-- Logs begin at Thu 2022-05-26 02:14:47 UTC, end at Thu 2022-05-26 03:43:22 UTC. -- May 26 03:03:46.046369 dv-cinder-843361-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 03:03:46.053723 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 03:03:46 2022] *** May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: nodename: dv-cinder-843361-1-iscsi May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: machine: x86_64 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: clock source: unix May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: pcre jit disabled May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: detected number of CPU cores: 4 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: current working directory: / May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: detected binary path: /usr/bin/uwsgi-core May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: your processes number limit is 47680 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: your memory page size is 4096 bytes May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: detected max file descriptor number: 2048 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: lock engine: pthread robust mutexes May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: thunder lock: enabled May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 03:03:46.057427 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 03:03:46.102026 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: Python main interpreter initialized at 0x555d6bb249c0 May 26 03:03:46.102026 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: python threads support enabled May 26 03:03:46.102026 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: your server socket listen backlog is limited to 100 connections May 26 03:03:46.102026 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: your mercy for graceful operations on workers is 90 seconds May 26 03:03:46.102402 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: mapped 403077 bytes (393 KB) for 2 cores May 26 03:03:46.102478 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: *** Operational MODE: preforking *** May 26 03:03:46.102728 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: *** uWSGI is running in multiple interpreter mode *** May 26 03:03:46.102728 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: spawned uWSGI master process (pid: 80404) May 26 03:03:46.102932 dv-cinder-843361-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 03:03:46.107264 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: spawned uWSGI worker 1 (pid: 80415, cores: 1) May 26 03:03:46.107264 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: spawned uWSGI worker 2 (pid: 80416, cores: 1) May 26 03:03:46.107264 dv-cinder-843361-1-iscsi devstack@keystone.service[80404]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 03:03:51.770521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 03:03:51.770521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes May 26 03:03:51.771349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 03:03:51.771349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: from cryptography.utils import int_from_bytes May 26 03:03:51.771489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 03:03:51.771489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes May 26 03:03:51.771917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 03:03:51.771917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: from cryptography.utils import int_from_bytes May 26 03:03:51.820742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 03:03:51.820882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 03:03:51.820970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 03:03:51.821048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 03:03:51.821109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] command line args: [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 03:03:51.821254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] command line args: [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 03:03:51.821341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 03:03:51.821500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 03:03:51.821585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ================================================================================ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 03:03:51.821655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ================================================================================ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 03:03:51.821755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.821905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.821984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_dir = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_dir = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] config_source = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] debug = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] config_source = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] debug = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.822943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.823931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_config_append = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_config_append = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_dir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_dir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.824943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.825833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.826032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.826108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.826229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.826311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.826428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.827909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.828669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] publish_errors = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] publish_errors = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.829958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] standard_threads = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] standard_threads = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.830950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_journal = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_json = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_journal = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_stderr = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_json = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_stderr = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] use_syslog = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.831969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] use_syslog = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.832046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.832196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 03:03:51.832279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.832449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.832535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.832682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.832758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.832935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.833950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.834908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.835865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.836980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.837958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838041 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.838941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.839878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.840963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.841948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.842976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.843905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.844998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.845994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.846926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.847887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.848848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.849962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.850904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.851939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.852906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.853978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.854954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.855899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.external = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.856999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.external = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.password = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.857992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] auth.token = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.password = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] auth.token = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.858963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.859942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.860875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.861980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.862992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.863972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.864928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.865880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.866922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.867914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.868990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.869992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.870938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.871917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.872986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.873064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.873720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.873804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.874976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.875922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.876925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.877853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.878940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.879947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.880903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.881979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.882973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.883944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.884853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.885944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.886957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.887917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.888768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.889481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.889577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.889675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.889825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.889904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.890882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.891851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.892971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.893929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.894989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.895873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.896839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.897954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.898968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.899926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.900861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.901962 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.902996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.903962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.904952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.905980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.906912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.907967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 03:03:51.908716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.908938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 03:03:51.909105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 03:03:51.939736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.939901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.939979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.940052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.940165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.940249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.948714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.948804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.977437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.977668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.978052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:51.978283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.031732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.031862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.083034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.083282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.083772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.084007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.148469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.148694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.148859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.148949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.149951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.173943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.174855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.186484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.186789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.198851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.199912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.200531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80416) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 03:03:52.219766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.219955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 03:03:52.220116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.220276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.220431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 03:03:52.220560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 03:03:52.220684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.220807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.220930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 03:03:52.221054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.221818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.232391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.233176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.233367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.233575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.233767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.233914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.234041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.234165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.241740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.242277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.245008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.245278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.245547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.245674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.245820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.245957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.246887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.247829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.248038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.248365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.274919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.275449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.276663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.276936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.277172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.277451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.283097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.283332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.283460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.283608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.283745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.283924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.290818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.291091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.291291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.291504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.295606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.295994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.296250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.296331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.296530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.296648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.296799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.296928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.297760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.298027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.298293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.316224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.316490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.316929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.317209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.321138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.321897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.324794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.325083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.325542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.325833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.330735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.331554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.334331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.335146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.338075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.338367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.338651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.338918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.339794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.340069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.340343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.359062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.359357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.359617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.359872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.360186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.360467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.360732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.360992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.367995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.369041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.369536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.369801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.370750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.371020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.372845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.373116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.373430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.373704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.373973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.374412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.374691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.374969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.375261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.375536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.387334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.387602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.387875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.388146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.388415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.388678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.389470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.389635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.389938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.390224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.390503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.390778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.391331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.391681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.407846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.409944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.411085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.413240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.414387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.416010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.416567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.417620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.417903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.418173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.418256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.419840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.420647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.420927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.426060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.427688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.428402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.430020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.431206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.431510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.431789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.432056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.433292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.433581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.433864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.434138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.441119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.441398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.441771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.443423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.443709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.444092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.449615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80416) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.449885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.450163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.450429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.450689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.450949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.451210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.452101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.452366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.452473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 03:03:52.452634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.452748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.452903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80416) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.453850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.454133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.454396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.454669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.454935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.455200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.455488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.455749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.456023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 03:03:52.493052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555d6bb249c0 pid: 80416 (default app) May 26 03:03:52.495172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555d6bb249c0 pid: 80415 (default app) May 26 03:03:52.496255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b6601-515f-4ef2-af05-14aee3e28aa2 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:03:52.496563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b6601-515f-4ef2-af05-14aee3e28aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:03:52.496839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b6601-515f-4ef2-af05-14aee3e28aa2 None None] PATH_INFO: `/v3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:03:52.501302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Thu May 26 03:03:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:00.422773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:00.423298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:00.423404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:00.427769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:00.432873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:00.433248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:00.433418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:00.503039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: INFO dbcounter [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] Registered counter for database keystone May 26 03:04:00.524181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG oslo_db.sqlalchemy.engines [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 03:04:00.530681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writer thread running {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 03:04:00.709562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80416) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 03:04:00.709985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG passlib.handlers.bcrypt [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80416) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 03:04:00.765866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:00.767737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:00.891844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 03:04:01.071130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:01.139792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: INFO dbcounter [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] Registered counter for database keystone May 26 03:04:01.159706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG oslo_db.sqlalchemy.engines [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80415) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 03:04:01.165893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writer thread running {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 03:04:01.285207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c945f77-d282-4542-a493-54c3ab7f8ac2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:01.289421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:01.290079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:01.290362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:01.290638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:01.299152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:01.350047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 03:04:01.350346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd304c6d-702c-4b67-97fd-615d5ddb93ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:01.360450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Thu May 26 03:04:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:04:01.366496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:01.381615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1940fb0-56c1-4497-b2fd-0da790538ede None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:01.386043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:01.387154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:01.387467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:01.387752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] PATH_INFO: `/v3/projects/fb9a8fcaeb99475786d754ce49440ef8` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:01.389275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] RBAC: Authorizing `identity:get_project(project_id=fb9a8fcaeb99475786d754ce49440ef8)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:01.442620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 03:04:01.443154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b7884-9c2f-4e00-9d28-a729ace73491 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:01.447656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Thu May 26 03:04:01 2022] GET /identity/v3/projects/fb9a8fcaeb99475786d754ce49440ef8 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:03.603112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d16a-834f-4894-a64f-0f492425f7aa None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:03.603639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d16a-834f-4894-a64f-0f492425f7aa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:03.603740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d16a-834f-4894-a64f-0f492425f7aa None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:03.607808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:03.613252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:03.613674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:03.613799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:03.632223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:03.633785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:03.642364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 03:04:03.664999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868d16a-834f-4894-a64f-0f492425f7aa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:03.677667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868d16a-834f-4894-a64f-0f492425f7aa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:03.681990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:03.682648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:03.682931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:03.683209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:03.694832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:03.696661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc647c2-f6e7-429d-bd8f-f31f3f1ca97e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:03.705765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Thu May 26 03:04:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 03:04:03.717421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:03.726694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26735232-eefc-417d-bfcb-6d7652ba204a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:03.731146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:03.732035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:03.732900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:03.733264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:03.734688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:03.736574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3d0ca-87d0-49b5-bb0d-002151c20506 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:03.738158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Thu May 26 03:04:03 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:06.012293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b7c2-6fd0-471c-814c-b852de00b8a9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.012820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b7c2-6fd0-471c-814c-b852de00b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.012908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b7c2-6fd0-471c-814c-b852de00b8a9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.013797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c76fd-5148-4d14-bc46-bd3f8123a753 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.014161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c76fd-5148-4d14-bc46-bd3f8123a753 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.014461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c76fd-5148-4d14-bc46-bd3f8123a753 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.014726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:06.015864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:06.020176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.020468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.020750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.020915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.021209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.021491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.038331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:06.039550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:06.060809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80415) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 03:04:06.061283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG passlib.handlers.bcrypt [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80415) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 03:04:06.107751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:06.109322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:06.118071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 7/12] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:06.140575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.164987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.211577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 6/13] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:06.235301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.245395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfa042-f0bb-453d-9b12-17da654d0f5c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.277873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.278455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.278735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.279579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.447495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a6bda-985f-4cb2-8195-9f9fd7c16566 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.449738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.465893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.467518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.485984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cce44-17f9-455a-9397-60715949502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.487027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cce44-17f9-455a-9397-60715949502d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.487351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cce44-17f9-455a-9397-60715949502d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.487640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cce44-17f9-455a-9397-60715949502d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.490244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.499645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cce44-17f9-455a-9397-60715949502d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-100cce44-17f9-455a-9397-60715949502d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:06.510591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:06.511671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:06.512660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:06.518318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cce44-17f9-455a-9397-60715949502d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.567852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 8/15] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 03:04:06.573261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.619194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cce44-17f9-455a-9397-60715949502d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.654616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.655306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.655587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.655858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.656608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:06.656914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.658359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.669967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde7541-3bbe-4058-a2fe-6a7203d98e0f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.670622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 8/16] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:06.675888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.692976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.704556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.704923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.705351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.705645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.705927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.716108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.717817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:04:06.741437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:06.742326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:06.743138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:04:06.743138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:04:06.743138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:04:06.743138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:06.743138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:06.749113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.778231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8ed3a-6814-473b-88ef-2a4ab38d588d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.812773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.813636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.813915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.814194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.815777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.817222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.818887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:06.824048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:06.848595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aed55-3c05-42a9-81e8-3d5f5adaa154 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.885030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.885690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.885971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.886246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.886974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:06.887306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.889399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.906627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:06.917043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:06.924914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82302d7-d7a2-4148-aba9-91f8ee215960 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:06.962713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:06.963659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:06.963942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:06.964213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:06.965478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:06.967458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24816776-5056-4456-9227-f81b0a0ebbc5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:06.969326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:06.975457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:07.014215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:07.085181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf650dca-e658-4a93-a88d-fd652a612522 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:07.120825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:07.121482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:07.121763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:07.122036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:07.123434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.domain.id=default, user.id=839fed83d5c84c7eb9422cc2681efe8f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:07.123841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:07.125304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c565-e88d-402f-9904-59dcf7efcbce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:07.132344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 11/21] 10.222.0.39 () {56 vars in 1322 bytes} [Thu May 26 03:04:06 2022] GET /identity/v3/role_assignments?user.id=839fed83d5c84c7eb9422cc2681efe8f&scope.domain.id=default&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:09.604314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af252-6cbc-4dd5-89c3-ff399ce3f7cf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:09.604837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af252-6cbc-4dd5-89c3-ff399ce3f7cf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:09.604837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35af252-6cbc-4dd5-89c3-ff399ce3f7cf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:09.606208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:09.612375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a03e-8718-478f-b78b-ada7b54bba0e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:09.612775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a03e-8718-478f-b78b-ada7b54bba0e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:09.612956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7a03e-8718-478f-b78b-ada7b54bba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:09.640967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9ae7a03e-8718-478f-b78b-ada7b54bba0e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:09.643360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ae7a03e-8718-478f-b78b-ada7b54bba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:09.740913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:09.759044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6d235-1926-4243-ab41-dadbfda4eebd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:09.759365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6d235-1926-4243-ab41-dadbfda4eebd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:09.759634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6d235-1926-4243-ab41-dadbfda4eebd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:09.760990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:09.775657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:09.775950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:09.776232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:09.777254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6d235-1926-4243-ab41-dadbfda4eebd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:09.806327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:09.807531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:09.869207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:09.976796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 13/25] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:10.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.024403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6d235-1926-4243-ab41-dadbfda4eebd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.104361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:10.105522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.106528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.107253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.131719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-13d39b42-7048-4524-841d-ca2439bab862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:10.145326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:10.146235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:10.146986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:10.150138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 13/26] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:10.152516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7715d88-ce6b-46aa-9946-0104d56499f7 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.154906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7715d88-ce6b-46aa-9946-0104d56499f7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.156529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7715d88-ce6b-46aa-9946-0104d56499f7 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.158630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 14/27] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:10.159739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7715d88-ce6b-46aa-9946-0104d56499f7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.215401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a920e6-f529-4b25-b299-c9da66555518 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.220623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:10.281603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:10.282193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.282472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.282746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.283522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:10.284924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf55a87-fd98-4ca6-a26d-ae05fac53f82 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:10.359054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7715d88-ce6b-46aa-9946-0104d56499f7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.425853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 14/28] 10.222.0.39 () {60 vars in 1131 bytes} [Thu May 26 03:04:10 2022] POST /identity/v3/roles => generated 219 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:04:10.427566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272eb4-ab33-45af-97a8-256bc313569f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.427861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272eb4-ab33-45af-97a8-256bc313569f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.428142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95272eb4-ab33-45af-97a8-256bc313569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.452767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:10.453477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.453757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.454036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.454863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:10.455177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:10.456636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:10.464371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-95272eb4-ab33-45af-97a8-256bc313569f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:10.467451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-95272eb4-ab33-45af-97a8-256bc313569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:10.475262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 15/29] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:10.478416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.492701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:10.642337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:10.668652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdfb1e-262b-4ae3-a619-d5882b4a5d32 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.673443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 15/30] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:10.675576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b0cf6-935c-4065-a1a8-89412cc1907f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.676050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b0cf6-935c-4065-a1a8-89412cc1907f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.676492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b0cf6-935c-4065-a1a8-89412cc1907f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.678101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 16/31] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:10.698861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.699428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.699846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.728130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:10.729038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.729317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.729652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.730954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:10.733587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:10.736897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 16/32] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:10 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:10.737883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.743011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:10.744260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:10.827940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 17/33] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:10.829184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.880080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f4b1f-780f-4c5b-bbf7-154737e6513c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.926907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:10.927661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:10.927942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:10.928319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:10.944072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:10.950847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:10.952837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:04:10.973452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:10.974852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:10.976492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:04:10.976492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:04:10.976492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:04:10.976492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:10.976884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd096263-6dd5-472f-b6a0-204043867953 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:10.977006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 17/34] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:10.977006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:10.985879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:11.016593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.017503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.017859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.018199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.019405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.020988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688a97c-1d43-46c4-9a4b-cf2186604f79 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.022614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 18/35] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:11.027471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749ff1-1041-46fd-b93a-5cead468de33 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.027471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749ff1-1041-46fd-b93a-5cead468de33 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.027471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749ff1-1041-46fd-b93a-5cead468de33 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.027471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 19/36] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:11.027471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60749ff1-1041-46fd-b93a-5cead468de33 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.097244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9178d6-ff54-4833-a376-ac9342252dd0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.132120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.132785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.133070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.133339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.134078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.135496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9fd0b-60a7-497f-8578-9bafcbf57185 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.140743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60749ff1-1041-46fd-b93a-5cead468de33 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.180033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.181015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.181483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.181957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.182947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:11.183478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.185246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.198872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 20/37] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:11.200026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.213570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:11.258980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 18/38] 10.222.0.39 () {60 vars in 1131 bytes} [Thu May 26 03:04:10 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:11.265707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.266322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.266620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.291969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:11.294210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:11.323226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971625a-3384-4168-9d9a-2f1b7d22d5e2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.365217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:11.374749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.375727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.379313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.379491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 19/39] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:11.379570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] PATH_INFO: `/v3/domains/default/users/839fed83d5c84c7eb9422cc2681efe8f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.379653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.381700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=839fed83d5c84c7eb9422cc2681efe8f, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.383435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f65a-412e-4e95-8336-c3747de9ce16 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.403598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 21/40] 10.222.0.39 () {58 vars in 1361 bytes} [Thu May 26 03:04:11 2022] PUT /identity/v3/domains/default/users/839fed83d5c84c7eb9422cc2681efe8f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:11.406075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf45266-1199-4e2f-abbe-86a69bee0189 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.406376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf45266-1199-4e2f-abbe-86a69bee0189 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.406656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf45266-1199-4e2f-abbe-86a69bee0189 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.408345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 22/41] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:11.409189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf45266-1199-4e2f-abbe-86a69bee0189 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.497663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:11.534438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7dcf1-91fd-4efe-9571-0517711d685a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.550869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf45266-1199-4e2f-abbe-86a69bee0189 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.577248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.578061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.578354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.578635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.579406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.581046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.590778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.592314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.592760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.593225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.594129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.595772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59966799-1600-43d8-8aac-ff074e9aabcc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.615431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 23/42] 10.222.0.39 () {60 vars in 1131 bytes} [Thu May 26 03:04:11 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:11.616927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.617294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.617653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.618403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 20/43] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:04:11 2022] POST /identity/v3/projects => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:11.638987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:11.640463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:11.737269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 24/44] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:11.761824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.819872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:11.879461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7118-752e-44ca-b66a-2c9fbc60fa3e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:11.917284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:11.918094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:11.918383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:11.918662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:11.920041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:11.921588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8935fce-0a6e-4f88-832e-dc59cad5934c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:11.923217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 21/45] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:11.928705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:11.984563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:12.038658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc78a-0d5d-4e51-baca-eca4b0492f0f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:12.072201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:12.072866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:12.073152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:12.073445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:12.074156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:12.075700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe695468-afc0-4b47-b86a-d1b4d70ee1ef None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:12.102064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 25/46] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:04:11 2022] POST /identity/v3/projects => generated 312 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:13.679096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01432913-839f-41d0-99f0-095bdc9c2e6b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:13.679635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01432913-839f-41d0-99f0-095bdc9c2e6b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:13.679716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01432913-839f-41d0-99f0-095bdc9c2e6b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:13.681040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 22/47] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:13.687046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:13.687473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:13.687649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:13.704669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:13.705884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:13.771177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 26/48] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:13.793620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01432913-839f-41d0-99f0-095bdc9c2e6b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:13.893726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01432913-839f-41d0-99f0-095bdc9c2e6b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:13.907667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:13.927154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:13.927980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:13.928264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:13.928536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:13.939122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:13.946594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:13.947601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:13.948425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 23/49] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:13.953577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:13.992553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:14.052258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972dd3-db98-4b09-8c85-551bdabf15ec None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.086060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.086722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.087003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.087320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.088081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:14.088386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.089774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.102029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 27/50] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:14.108515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:14.212059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169846f1-5547-41ca-b1d3-37266b630d3d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.245184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.246047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.246328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.246603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.248643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.250102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592cbdb1-055d-4704-a79c-882c7291b91c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.251764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 24/51] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:14 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:14.257188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:14.315852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084095f-6cc4-45f9-8722-289f7373114b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.316148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084095f-6cc4-45f9-8722-289f7373114b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.316432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084095f-6cc4-45f9-8722-289f7373114b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.317798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 25/52] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:14.322565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.322852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.323134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.339406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:14.340595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:14.354715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:14.358713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df4bce-432a-4b04-9808-f90d6fdc3266 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.378139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:14.401093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.401895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.402176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.402447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.404196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.405694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.407332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 28/53] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:14.411123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 26/54] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:14.414057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:14.434710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:14.518977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f2cd-2729-48fc-ab84-9e83a82c5fe5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.534358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba14d2-cc8c-4fae-9abc-90e796c9c2ee None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.552757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.553407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.553692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.553971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.554827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.domain.id=default, user.id=839fed83d5c84c7eb9422cc2681efe8f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:14.555139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.556616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.563097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 29/55] 10.222.0.39 () {56 vars in 1322 bytes} [Thu May 26 03:04:14 2022] GET /identity/v3/role_assignments?user.id=839fed83d5c84c7eb9422cc2681efe8f&scope.domain.id=default&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:14.568155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.568967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.569250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.569521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.570734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.572262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee70268-0404-4397-b20f-f9e83a2c6bfd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.573966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 27/56] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:14.580531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:14.615742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:14.688749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463958fd-d29f-4a86-ad13-19ff2c845058 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:14.724418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:14.725104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:14.725390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:14.725666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:14.726387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:14.727905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ff3e-3e0b-4eda-a2fc-44a7895b06e3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:14.756701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 30/57] 10.222.0.39 () {60 vars in 1140 bytes} [Thu May 26 03:04:14 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:17.111123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9ebaa-a8c3-480f-a78a-dc5512f7ec42 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:17.111672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9ebaa-a8c3-480f-a78a-dc5512f7ec42 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:17.111785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9ebaa-a8c3-480f-a78a-dc5512f7ec42 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:17.113134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 28/58] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:17.118327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:17.118744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:17.118894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:17.135830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:17.137020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:17.201971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 31/59] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:17.226320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9ebaa-a8c3-480f-a78a-dc5512f7ec42 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:17.269952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:17.327673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9ebaa-a8c3-480f-a78a-dc5512f7ec42 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:17.360254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:17.361031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:17.361364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:17.361591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:17.362750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:17.364254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9a600-0d0a-43c0-9480-6d4b3afc3948 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:17.365899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 29/60] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:17.371396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:17.386454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:17.467599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c8fa0-f5eb-463f-9778-a2d0cd4c9220 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:17.502414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:17.503074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:17.503388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:17.503666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:17.504379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:17.505821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6a35a-3467-4a5b-8fba-3a148d7cc4c7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:17.530770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 32/61] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:17 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:20.996957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010fb26-a825-4050-835a-4fd400bf495c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:20.998379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010fb26-a825-4050-835a-4fd400bf495c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:20.998886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010fb26-a825-4050-835a-4fd400bf495c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.000492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 30/62] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:21.006218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4f92-8e51-4fa0-98a5-14fbaceaf9d3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.007070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4f92-8e51-4fa0-98a5-14fbaceaf9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.007562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a4f92-8e51-4fa0-98a5-14fbaceaf9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.031684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-972a4f92-8e51-4fa0-98a5-14fbaceaf9d3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:21.033226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-972a4f92-8e51-4fa0-98a5-14fbaceaf9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:21.084819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1311232-3e17-4d0f-90db-0935a21dfaba None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.085452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1311232-3e17-4d0f-90db-0935a21dfaba None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.085924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1311232-3e17-4d0f-90db-0935a21dfaba None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.087551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 31/63] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:21.093610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.095071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.095612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.115120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:21.116673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:21.139979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.147266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 33/64] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:21.149759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780d87e-d688-449e-bc29-8ce7f3a58ab0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.150311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780d87e-d688-449e-bc29-8ce7f3a58ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.150825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780d87e-d688-449e-bc29-8ce7f3a58ab0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.153914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 34/65] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:21.159283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.159283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.159283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.176873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:21.178406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:21.204918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 32/66] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:21.206060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.259571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.283285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 35/67] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:21.288205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.347412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e03d-bb21-43ec-a8bd-31c0781c6219 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.394165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.395277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.395710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.396132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.408686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:21.409544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bca55-93f5-420c-8124-7625efac51c2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:21.416825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:21.417670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:21.418429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 33/68] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:21.419146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.440116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.446454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.447432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.447725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.448006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.460859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-34697682-83de-4443-b1b6-0c3a30f79771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:21.470078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:21.470967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 36/69] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:21.476631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5e0d-9e96-449e-9b9d-c47289d5f008 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.477304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5e0d-9e96-449e-9b9d-c47289d5f008 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.477304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5e0d-9e96-449e-9b9d-c47289d5f008 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.477304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 37/70] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:21.477304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc5e0d-9e96-449e-9b9d-c47289d5f008 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.533386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.539743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a92-ad9e-41a7-8eee-aedee25cc6db None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.575026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.576431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.576709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.576979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.582607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc5e0d-9e96-449e-9b9d-c47289d5f008 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.586724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:21.594240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:21.595121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:21.595924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 34/71] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:21.596753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.621050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.621722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.622620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.622918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.623874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:21.624187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:21.625597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc950f-534f-4f37-8df2-9638abe39101 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:21.633534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 38/72] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:21.634718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159faeb-ee47-4c2b-a922-17d9a1bb5508 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.635009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159faeb-ee47-4c2b-a922-17d9a1bb5508 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.635315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159faeb-ee47-4c2b-a922-17d9a1bb5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.652990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f159faeb-ee47-4c2b-a922-17d9a1bb5508 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:21.654820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f159faeb-ee47-4c2b-a922-17d9a1bb5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:21.696150 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6290f-58cb-4c27-8eb3-3d7165b1919a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.701731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.722964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 39/73] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:21.724288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1912d5-8e76-4876-89a7-0e3088eee9bf None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.724578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1912d5-8e76-4876-89a7-0e3088eee9bf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.724861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1912d5-8e76-4876-89a7-0e3088eee9bf None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.726133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 40/74] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:21.726828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1912d5-8e76-4876-89a7-0e3088eee9bf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.739466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.740139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.740427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.740706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.741457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:21.741769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:21.743157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:21.752392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 35/75] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:21.753203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.776945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.833234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1912d5-8e76-4876-89a7-0e3088eee9bf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.854667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b786e26-4c97-40eb-8e69-c46fb94c93f2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:21.866410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.867070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.867395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.867678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.868436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:21.868749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:21.870158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf561ba-03db-4925-9ac9-f43084671cdd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:21.878395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 41/76] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:21.879671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.879968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.880255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.892717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:21.893601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:21.893868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:21.894144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:21.896010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:21.897229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:21.910118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:21.911617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:21.913286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 36/77] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:21.914044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.939169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:21.966877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 42/78] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:21.975363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:21.996265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.017939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339ef02-4a41-406f-8de0-dcc064af8f32 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.051410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.052210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.052499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.052770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.053943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.055430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.057042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 37/79] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:22.057796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.076365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b56fcf-2900-47eb-9e18-69c112e7d515 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.110480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.111405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.111754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.112106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.113357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.114893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.116621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 43/80] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:21 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:22.117380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.155984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b35721-4539-4871-b4c1-e5b5dd60ea84 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.162508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.190790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.191686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.191972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.192250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.193448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.194909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.196580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 38/81] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:22.197343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.216073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.220315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b4b3-af67-4225-bcf9-fbc79c0d7a5b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.254311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.254960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.255259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.255560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.298998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7b5ed-99bc-493d-a458-92b8945f49a3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.321029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.322509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.332117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.332897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.333177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.333448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.344072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:22.351764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:22.352688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:22.353438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 39/82] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:22.354056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.368702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.426047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 44/83] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:22.426881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.430126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.457318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c59c5d-faa4-48ba-a112-edc41f08dbc6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.490012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.490673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.490957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.491257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.491961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.493407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.519388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 40/84] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:22 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:22.520201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.527280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca411c-0af3-430e-bf49-37affaa89a4d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.565448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.566104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.566388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.566666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.580397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.639145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44ebd-63ff-4e8c-8c01-d75ce0219840 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.642813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.644357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.673871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.683006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.684229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.685266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.686066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.760021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 45/85] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:22.760886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.769370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.771171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.821979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.891561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 41/86] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:22.892451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:22.892853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879f0c8-a607-49c0-959d-10c4e2473682 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:22.931686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:22.931875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:22.932807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:22.933091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:22.933362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:22.934122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:22.934956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:22.936651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:22.953664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 46/87] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:22.954902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.012173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333299-d75e-4f2b-945f-eb6ab5fe993e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.058279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.059509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.059805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.060088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.061486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.063292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.065644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 42/88] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.066693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.080218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af6709-9ccd-4a7d-92c9-e537a5e143c0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.083816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:23.133061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.134469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.134899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.135354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.136885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.139652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.142869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 47/89] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:22 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.147417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.191822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:23.196583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1542f-0542-4fd4-ba64-8667eb23f89b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.236229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.237307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.237820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.238154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.239609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.241227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.243285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 43/90] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.244117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.255418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc31b6a-c23c-4df8-9917-1a21aadf7f12 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.290376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.291610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.292093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.292565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.294522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.296357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.298240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 48/91] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.299216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.341991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:23.352096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2f13-277f-4894-bef6-511ba7830487 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.400198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.400987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.401211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.401495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.402349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:23.402643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.405307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234bdd5-3b5b-4b84-b5d4-9a4556f75c16 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.405464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.416660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 44/92] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=986616b48b464a8c82fe68f806417c4a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.417523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.451679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.452732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.453274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.453769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.453983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:23.455010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:23.455554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.457265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.464046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 49/93] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=29fc77f10237449d82dfef29036abe51 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.465326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:23.579362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c96c-2198-477a-99b1-b7704e252269 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.580096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30e70-a585-497e-929e-af627c762889 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:23.582981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:23.618191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.619308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.619795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.620270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.643995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:23.644676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:23.644964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:23.645243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:23.646092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=839fed83d5c84c7eb9422cc2681efe8f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:23.646406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.648272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.664170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 45/94] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/role_assignments?user.id=839fed83d5c84c7eb9422cc2681efe8f&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:23.709521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:23.712134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:23.843354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 50/95] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:23.849105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:24.002686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:24.009504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41784f9-127a-4210-8ea8-8d9faadc2e2d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:24.056877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:24.058086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:24.058549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:24.060010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] PATH_INFO: `/v3/projects/5b027b887126434fa4af94940dfb61bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:24.061526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] RBAC: Authorizing `identity:get_project(project_id=5b027b887126434fa4af94940dfb61bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:24.063183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bce736-9e87-4a20-aa7b-798030e74e59 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:24.065145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 46/96] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:23 2022] GET /identity/v3/projects/5b027b887126434fa4af94940dfb61bc => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:24.071193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:24.074685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:24.191005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642032-d042-4475-a88a-5e987319faa5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:24.227060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:24.228041 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:24.228500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:24.228980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:24.229989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=5b027b887126434fa4af94940dfb61bc, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:24.230476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:24.232267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21236a31-d61d-403b-98d0-bf1f33a526a8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:24.238560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 51/97] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:24 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=5b027b887126434fa4af94940dfb61bc&role.id=29fc77f10237449d82dfef29036abe51 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:25.450917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657e86-29ce-4234-9b53-b23dbd3aa827 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.451992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657e86-29ce-4234-9b53-b23dbd3aa827 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.452489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657e86-29ce-4234-9b53-b23dbd3aa827 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:25.454027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 47/98] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:25.455473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bd4e-45a9-400f-b52c-a43adb4e33b5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.455862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bd4e-45a9-400f-b52c-a43adb4e33b5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.456037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bd4e-45a9-400f-b52c-a43adb4e33b5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:25.457427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 52/99] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:25.460592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.460891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.461177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:25.468633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.468919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.469204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:25.491126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:25.492366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:25.505139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:25.506331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:25.601259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 48/100] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:25.629650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:25.630065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:25.636732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:25.671881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:25.858718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:25.879509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980519bb-2a89-4abc-b449-004924a2eccb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:25.897339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a82df-f38c-47a1-b713-24232f5831f5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:25.952059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:25.953440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.953920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.954436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:25.961117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:25.963847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:25.966370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 49/102] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:25.972593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:25.982522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:25.983279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:25.983569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:25.983850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.009414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:26.018496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:26.019439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:26.020206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:26.022910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 54/103] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:26.028864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:26.136717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:26.190211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0120ec4-a028-41c5-84f9-a0d352d51923 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:26.223143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cc10-ff75-4042-be5e-64ad586b26c0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:26.241431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:26.242105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.242392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.242670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.243414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:26.244861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:26.286205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:26.286909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.291348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.291690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.291796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 50/104] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:04:25 2022] POST /identity/v3/projects => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:26.292498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:26.292823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:26.294274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:26.312227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 55/105] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:26.318111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:26.343653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:26.534786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa3552-192d-4176-a8cb-6e618a2bfddd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:26.574726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:26.575690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.575978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.576256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] PATH_INFO: `/v3/users/9414200cd2984db691d6df7b34aa5d65` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.591838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9414200cd2984db691d6df7b34aa5d65)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:26.593342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e23c31-b79f-4ec1-91a7-3c42dcf342d9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:26.595095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 51/106] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:26 2022] GET /identity/v3/users/9414200cd2984db691d6df7b34aa5d65 => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:26.600787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:26.723081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b78de0-0f6e-472d-9728-5d5386671850 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:26.750644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:26.770787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:26.771486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.771774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.772072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.796173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6dbd-2fbb-4dd1-839c-8d5d8fae1c4b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.796357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6dbd-2fbb-4dd1-839c-8d5d8fae1c4b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.796641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6dbd-2fbb-4dd1-839c-8d5d8fae1c4b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.798052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 52/107] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:26.816820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:26.817125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:26.817380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:26.848790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:26.849996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:26.908390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:26.909895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:26.974534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:26.986307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 53/108] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:27.044628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 56/109] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:27.050561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.055365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.094498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:27.259772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228c31c-fb45-4627-b5fe-0f963e4b3ad9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.293958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595d417-df99-4605-aca9-11c9ca48864d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.329501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.331071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.331596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.332085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.334030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:27.335922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f37a6-8143-4f06-ba45-1636bb9a59fe None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:27.337891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 54/110] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:27.339482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175eb31-be53-4efe-9e2b-2badec3e0cd4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.340184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175eb31-be53-4efe-9e2b-2badec3e0cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.341341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175eb31-be53-4efe-9e2b-2badec3e0cd4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.343433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 55/111] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:27.348192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175eb31-be53-4efe-9e2b-2badec3e0cd4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.378781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.379621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.379910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.380183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.392243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0d860096-5528-4fee-8ac8-a7c1e00dd5d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:27.399998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:27.400842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:27.401584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 57/112] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:27.402742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd1539-72ef-4071-be79-4dadf47f17cf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.402742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd1539-72ef-4071-be79-4dadf47f17cf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.403519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd1539-72ef-4071-be79-4dadf47f17cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.420879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e0bd1539-72ef-4071-be79-4dadf47f17cf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:27.422079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0bd1539-72ef-4071-be79-4dadf47f17cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:27.448564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:27.492159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175eb31-be53-4efe-9e2b-2badec3e0cd4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.503082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 58/113] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:27.508728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48800a-6538-40ad-ad4b-85be3c5ff49d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.509024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48800a-6538-40ad-ad4b-85be3c5ff49d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.509308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48800a-6538-40ad-ad4b-85be3c5ff49d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.511068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 59/114] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:27.511833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48800a-6538-40ad-ad4b-85be3c5ff49d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.534116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:27.545051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.546330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.546623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.547201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.548172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=9414200cd2984db691d6df7b34aa5d65) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:27.548642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:27.550769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50baf5-108b-4d10-989d-df5319b22c09 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:27.558334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 56/115] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/role_assignments?user.id=9414200cd2984db691d6df7b34aa5d65&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:27.565230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c265c1-d100-4c53-a2fc-a2dae219b833 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.565230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c265c1-d100-4c53-a2fc-a2dae219b833 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.565493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c265c1-d100-4c53-a2fc-a2dae219b833 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.566812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 57/116] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:27.568034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.568321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.568663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.587552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:27.589705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:27.653590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48800a-6538-40ad-ad4b-85be3c5ff49d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.690247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 58/117] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:27.691610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.702780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.703581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.703874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.704159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.705330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:27.705457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:27.706878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80fd44-69d0-4944-acc7-e6343a0ada69 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:27.729717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 60/118] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:27.731448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.731907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.732195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.753114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:27.754370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:27.769996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:27.814280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d278e-6082-456b-b07a-d559c91d2b92 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.827213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 61/119] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:27.828390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.830755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:27.849515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.850306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.850598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.850879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.861851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:27.869837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:27.870737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:27.871614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 59/120] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:27.873325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:27.943725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1096-ab32-4bb6-baeb-61180172c1fd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.980194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30dc50e-06d4-4b1b-95d6-445ba9aac41a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:27.981682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:27.982746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:27.983187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:27.983686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:27.985096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:27.986751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:27.988589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 62/121] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:27.989568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.005963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.020012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.021087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.021576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.022049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.034065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:28.042495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:28.043389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:28.044474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:28.047096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 60/122] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:28.048136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.065520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.100944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07922f01-b733-4adb-841c-04cff5182fda None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.137810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.138966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.139513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.139986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.151100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:28.160191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:28.161026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:28.161787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:28.162983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dafa9-4387-4cbf-a60b-88091dba6682 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.164433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 63/123] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:28.165576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.205699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.206378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.206666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.206945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.207737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:28.208050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.209918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.225450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 61/124] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.225450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.274846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.292038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccd36-bf67-46c8-ab82-271e21e0c651 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.333088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709cf1a-9df8-4472-8737-90cde1af1289 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.335910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.338681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.339706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.340067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.340436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.342161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.344323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.347017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 64/125] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.348543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.383794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.384575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.385170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.385303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.386045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:28.386420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.388724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.397772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 62/126] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.399175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.477802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75592ee-f144-49f6-abb4-228f05970bb1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.516860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.517673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.518024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.518375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.519203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:28.519619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.521145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.525138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe86e9-279c-474e-b7ea-b6ed791020c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.530224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 65/127] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.530904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.535468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.572289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.574148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.575596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.576103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.577528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.579185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.581800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 63/128] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.582772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.607101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.652445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7c474-3058-4ea4-8bfb-5783902226c4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.690789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.692485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.692953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.693442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/033961e12b9349dbba61d19814615bd3/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.695687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, user_id=033961e12b9349dbba61d19814615bd3, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.697332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.714080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 66/129] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:28 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/033961e12b9349dbba61d19814615bd3/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:28.716278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.724643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc62d-01f6-4b59-a863-bcc5144f1ccb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.768927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be038214-433d-4796-a35b-304afdab6014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.770036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be038214-433d-4796-a35b-304afdab6014 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.770452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be038214-433d-4796-a35b-304afdab6014 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.770831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be038214-433d-4796-a35b-304afdab6014 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.773028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be038214-433d-4796-a35b-304afdab6014 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.774295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be038214-433d-4796-a35b-304afdab6014 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.776317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 64/130] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.777317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be038214-433d-4796-a35b-304afdab6014 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.832144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.855256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3812e2-3fce-49bb-8b13-0c8e84764829 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.882489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:28.902005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.903525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.903799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.904052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.905273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.905854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be038214-433d-4796-a35b-304afdab6014 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:28.906752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.909763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 67/131] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.910587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:28.954667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:28.955577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:28.955863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:28.956141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:28.957339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:28.959376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:28.961727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 65/132] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:28.962360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:29.044704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a6a4-5864-4c31-8e80-1313158f0aa2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.083140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.084052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.084337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.084622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.085869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.087427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.090007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 68/133] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:29.091627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:29.132154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3a6c0-7d87-4245-a382-8b65e0e63d88 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.145154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:29.158291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:29.174161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.175043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.175352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.175635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] PATH_INFO: `/v3/projects/5b027b887126434fa4af94940dfb61bc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.176916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] RBAC: Authorizing `identity:get_project(project_id=5b027b887126434fa4af94940dfb61bc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.178433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.180489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 66/134] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:28 2022] GET /identity/v3/projects/5b027b887126434fa4af94940dfb61bc => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:29.181075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:29.206125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6308b44-79b1-48c9-93d7-aedbee240eda None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.241765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.242801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.243157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.243529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/839fed83d5c84c7eb9422cc2681efe8f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.245586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, user_id=839fed83d5c84c7eb9422cc2681efe8f, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.247664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.273265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 69/135] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:29 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/839fed83d5c84c7eb9422cc2681efe8f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:29.274352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:29.307014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0af7bc-c0f7-404d-ab90-4b8f0bca06e7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.385450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.386349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.386761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.387096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/033961e12b9349dbba61d19814615bd3/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.396162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, user_id=033961e12b9349dbba61d19814615bd3, role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.397665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decec9cf-2cb8-4f91-a8f5-6987ecc00c7c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.431101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 67/136] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:29 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/033961e12b9349dbba61d19814615bd3/roles/986616b48b464a8c82fe68f806417c4a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:29.472637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d31f5-8ed3-4f0f-8360-081b45d08e1a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.472956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d31f5-8ed3-4f0f-8360-081b45d08e1a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.473309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d31f5-8ed3-4f0f-8360-081b45d08e1a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.474778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 68/137] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:29.476329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e2d1-578c-4de5-b232-b390406cdc7e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.490567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.490888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.491195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.495902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:29.517963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.518929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.519213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.519552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] PATH_INFO: `/v3/projects/5b027b887126434fa4af94940dfb61bc/users/033961e12b9349dbba61d19814615bd3/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.527163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b027b887126434fa4af94940dfb61bc, user_id=033961e12b9349dbba61d19814615bd3, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.530258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:29.531522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.534053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:29.560407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 70/138] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:29 2022] PUT /identity/v3/projects/5b027b887126434fa4af94940dfb61bc/users/033961e12b9349dbba61d19814615bd3/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:29.606590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:29.661778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 69/139] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:29.709825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:29.837262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908ed49-7c88-4123-96a3-f05de1905aa0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:29.874823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:29.875670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:29.875957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:29.876235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:29.877439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:29.878919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b242f9-9c91-45a1-8d1e-b56781f369b1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:29.880589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 71/140] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:29.886331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:30.038627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff219532-09bc-4379-afc6-e7f01f5610a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:30.086125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:30.086850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:30.087142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:30.087463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:30.088187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:30.089641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298196-8d1c-42d6-af85-770b1d83dade None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:30.119204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 70/141] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:29 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:30.349403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be60cd-5c9f-4998-ab8c-4bb72576f618 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:30.349583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be60cd-5c9f-4998-ab8c-4bb72576f618 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:30.349862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be60cd-5c9f-4998-ab8c-4bb72576f618 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:30.351697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 72/142] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:30.358158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:30.358448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:30.358729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:30.384586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:30.385778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:30.412485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:30.489559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 71/143] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:30.516651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be60cd-5c9f-4998-ab8c-4bb72576f618 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:30.579828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:30.656537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be60cd-5c9f-4998-ab8c-4bb72576f618 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:30.701007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:30.702139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:30.702649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:30.703141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:30.717270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:30.725718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:30.726566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:30.727369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:30.729710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 73/144] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:30.739918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:30.884794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0c3d1-7a7a-4cac-b1f6-238b3a63e2db None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:30.928012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:30.928612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:30.928894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:30.929173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:30.929928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:30.930235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:30.931660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:30.940718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 72/145] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:30.946648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:31.062150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b148-0a4a-40ad-a121-e7d2470f16c0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:31.068277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:31.108679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:31.109561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:31.109842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:31.110121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] PATH_INFO: `/v3/users/9414200cd2984db691d6df7b34aa5d65` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:31.111374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] RBAC: Authorizing `identity:get_user(user_id=9414200cd2984db691d6df7b34aa5d65)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:31.112859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:31.114620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 74/146] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:30 2022] GET /identity/v3/users/9414200cd2984db691d6df7b34aa5d65 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:31.120586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:31.141995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:31.247718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9a81a-3e41-4194-bb2b-7552b1861a5d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:31.295524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:31.296405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:31.296704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:31.296979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:31.298298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:31.299815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee33-3f5d-40f9-a4f5-68730c0798cf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:31.301533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 73/147] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:31 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:31.307441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:31.440793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8567337-ac83-4f0f-95f0-c30d4f5885a5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:31.477594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:31.478493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:31.478773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:31.479049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/9414200cd2984db691d6df7b34aa5d65/roles/3c73ee9c7315428098a4e3ef35aad1ea` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:31.481922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, user_id=9414200cd2984db691d6df7b34aa5d65, role_id=3c73ee9c7315428098a4e3ef35aad1ea)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:31.483417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a4144-c5b2-4f3e-8973-fc8573b0e9ee None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:31.501562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 75/148] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:31 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/users/9414200cd2984db691d6df7b34aa5d65/roles/3c73ee9c7315428098a4e3ef35aad1ea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:32.304921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5007c1-b776-4e80-bd24-89f2fdff7030 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.305935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5007c1-b776-4e80-bd24-89f2fdff7030 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.306444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5007c1-b776-4e80-bd24-89f2fdff7030 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.308122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 74/149] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:32.314803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.315736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.316228 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.344075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:32.345604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:32.391186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:32.446177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 76/150] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:32.483788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5007c1-b776-4e80-bd24-89f2fdff7030 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:32.597462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:32.620859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5007c1-b776-4e80-bd24-89f2fdff7030 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:32.666744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:32.667576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.667863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.668142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.685005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:32.692738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:32.693675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:32.694443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 75/151] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:32.701190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:32.816443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf266f-d58d-4cea-b2fe-ed45c0908d21 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:32.853121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:32.853729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.854011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.854287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.855059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:32.855401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:32.857399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342af0be-84a5-43aa-86de-d57a8c128bc9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:32.866996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 77/152] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:32.873501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:32.884815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3980-fb7b-4dab-8560-ed94cdc67fdc None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.885105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3980-fb7b-4dab-8560-ed94cdc67fdc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.885384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3980-fb7b-4dab-8560-ed94cdc67fdc None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.886754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 78/153] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:32.896074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:32.896363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:32.896644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:32.938479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:32.939727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:32.981399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.046170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 79/154] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:33.076117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b74eb-9846-4f0d-9bab-02795cb87d50 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.092726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.130487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.131404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.131684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.131961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.135993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:33.137634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:33.139653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 76/155] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:32 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:33.145340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.171845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.214854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37362a-1a1e-4432-bd92-30023970b24d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.277078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.278841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.279138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.279138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.301014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f48500-6686-43bd-b571-e2e83c03d631 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.312510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-54f04e62-81ad-4fe8-b3a3-145e2e3dcbdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:33.327278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:33.328775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:33.329626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:33.332323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 80/156] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:33.333840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9794-c928-4ff3-a36f-5538df89e767 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.334282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9794-c928-4ff3-a36f-5538df89e767 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.334700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c9794-c928-4ff3-a36f-5538df89e767 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.335801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.336523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.336895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 81/157] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:33.337001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.341682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.343612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.344110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.344608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.384482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:33.386032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:33.407267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.411775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:33.413490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:33.439614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.514508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 82/158] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:33.515639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.538244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 77/159] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 03:04:33.544570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.717168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010cf6-1463-4731-acc7-4b186147f0a9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.728941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a698bc-d4fa-4b04-a5b4-ba1bc9adcb8c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.749403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.784313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.786555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.786838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.787116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.787902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:33.788204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:33.789591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58142a-f09d-472c-940c-507c8885beb4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:33.794438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.795502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.795783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.796055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.799914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:33.801357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:33.803046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 78/160] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:33.803827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.809632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 83/161] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:33.810868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a5db-0163-4a73-8312-1916ab792777 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.811118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a5db-0163-4a73-8312-1916ab792777 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.813066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a5db-0163-4a73-8312-1916ab792777 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.819464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 84/162] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:33.820203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a5db-0163-4a73-8312-1916ab792777 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:33.834165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:33.942081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1409-5267-4395-83b1-a60bf88fff73 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:33.977824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:33.978948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:33.979482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:33.979977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:33.991387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:33.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:34.000404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:34.001171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:34.003506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 79/163] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:34.004491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.011987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a5db-0163-4a73-8312-1916ab792777 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.096031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.097048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.097472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:34.098828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.101405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.103178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:34.103851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:34.108380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4484424-e0d9-4f56-b532-6062fd719385 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:34.122445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 85/164] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:33 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=29fc77f10237449d82dfef29036abe51 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:34.125228 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.125855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.127695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.168836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:34.172197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:34.233187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7ce09-d246-4335-9ae4-406dc29a0e09 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.299204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:34.311159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 86/165] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:34.312178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.313115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.313246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.313620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.313997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] PATH_INFO: `/v3/users/839fed83d5c84c7eb9422cc2681efe8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.315408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] RBAC: Authorizing `identity:get_user(user_id=839fed83d5c84c7eb9422cc2681efe8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:34.316970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:34.318723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 80/166] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/users/839fed83d5c84c7eb9422cc2681efe8f => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:34.332275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.484413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3455e-fe71-4e6a-af34-c8cb6a86e390 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.495604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:34.521877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.522967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.523375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.523778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.537707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13229e-2827-4641-9f67-ae5fe2a64b4b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.596937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:34.597293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.598368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.598660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.598938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.599181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:34.599763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:34.600077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:34.601564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:34.621080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 87/167] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:34.622323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.714702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 81/168] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 03:04:34.717100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.719434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:34.727602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:34.770254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a235a-9212-44b8-9c7f-11162e8a4cd4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.814026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.814843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.815149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.815430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.827745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:34.835869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:34.838015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:34.840017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 88/169] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:34.841886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb77ae-9cdf-4470-ad1c-492f7174f1e9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.842030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.897617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:34.898497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:34.898777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:34.899052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:34.902182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:34.903877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:34.905794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 82/170] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:34.906567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:34.959997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4807fc-ff3c-4e67-88a0-beef1fcff42c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:34.986135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.000453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.001328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.001613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.001889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.003096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.004608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.006616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 89/171] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:35.011547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:35.031687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.039820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb84dd-8689-4759-a3dd-81b383bcddec None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:35.077366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.078086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.078370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.078644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.079446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:35.079753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.081129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.089938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 83/172] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:35.090720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:35.140008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ede663-b19a-4c3b-89d8-eef5b9e5dcbd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:35.208484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.209198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.209476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.209749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.224962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebb387-c41c-4709-ab16-a0e6c10e263f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:35.307463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.308130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.308535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.309644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.310649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=839fed83d5c84c7eb9422cc2681efe8f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:35.311131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.312729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.333134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 84/173] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:35 2022] GET /identity/v3/role_assignments?user.id=839fed83d5c84c7eb9422cc2681efe8f&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:35.334080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:35.347636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.371643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.393985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.395472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c8cc9-5fdf-4808-8f6f-94bbb2b5eb33 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.547217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d884f1-8448-423a-818e-f445a2c81cbf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:35.619375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.619375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.621315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.622056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] PATH_INFO: `/v3/users/033961e12b9349dbba61d19814615bd3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.628214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] RBAC: Authorizing `identity:get_user(user_id=033961e12b9349dbba61d19814615bd3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.629957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d04a-ff6f-4b51-862f-2fc9ca25b241 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.632376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 85/174] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:35 2022] GET /identity/v3/users/033961e12b9349dbba61d19814615bd3 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:35.633921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090637e6-09a9-4eb4-bbbe-b7b0d8dd67d3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.634394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090637e6-09a9-4eb4-bbbe-b7b0d8dd67d3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.634839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090637e6-09a9-4eb4-bbbe-b7b0d8dd67d3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.639470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 86/175] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:35.641390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8e96-b049-41df-be6c-567f5069fd3a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.641880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8e96-b049-41df-be6c-567f5069fd3a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.642338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e8e96-b049-41df-be6c-567f5069fd3a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.643737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 87/176] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:35.644661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e8e96-b049-41df-be6c-567f5069fd3a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:35.670843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 90/177] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:35.676843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104c462-a274-49ea-89d9-b45ec8eccef9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.677289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104c462-a274-49ea-89d9-b45ec8eccef9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.677715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104c462-a274-49ea-89d9-b45ec8eccef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.696594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a104c462-a274-49ea-89d9-b45ec8eccef9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:35.698017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a104c462-a274-49ea-89d9-b45ec8eccef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:35.733185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.773265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e8e96-b049-41df-be6c-567f5069fd3a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:35.782991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 91/178] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:35.784692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.785160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.785605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.803695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:35.805176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:35.813873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:35.814869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:35.815341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:35.815764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:35.821705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:35.880122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 92/179] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:35.881039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:35.916527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:35.918031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:35.994113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fe586-ae53-44bc-a6a0-263b5c0a49ef None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.003767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.035175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.036095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.036374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.036643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] PATH_INFO: `/v3/projects/5b027b887126434fa4af94940dfb61bc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.037833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] RBAC: Authorizing `identity:get_project(project_id=5b027b887126434fa4af94940dfb61bc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.039419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.041281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 93/180] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:35 2022] GET /identity/v3/projects/5b027b887126434fa4af94940dfb61bc => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.041877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.052667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 88/181] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:36.053743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.065386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.154886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e665d2-d533-4ad3-97c8-d744722d38f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.160711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e5a9-efbe-4fd4-a877-3abd2e1f4802 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.189054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.189840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.190116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.190382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.191565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.193068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f145e-76bc-48be-946c-226dcd35d204 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.194126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312baff-c751-4e94-b977-a54781330836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.194629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 94/182] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.199108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312baff-c751-4e94-b977-a54781330836 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.199108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312baff-c751-4e94-b977-a54781330836 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.199108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312baff-c751-4e94-b977-a54781330836 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.199355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729794-2524-4ef8-90d3-078897fda289 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.199355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729794-2524-4ef8-90d3-078897fda289 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.199355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729794-2524-4ef8-90d3-078897fda289 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.199355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 95/183] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:36.199355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729794-2524-4ef8-90d3-078897fda289 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.210670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312baff-c751-4e94-b977-a54781330836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d312baff-c751-4e94-b977-a54781330836 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:36.219141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:36.220204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:36.221001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 89/184] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:36.221647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312baff-c751-4e94-b977-a54781330836 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.252399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.284956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.301536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729794-2524-4ef8-90d3-078897fda289 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.320246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312baff-c751-4e94-b977-a54781330836 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.334675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.335347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.335624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.335888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.336713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=5b027b887126434fa4af94940dfb61bc, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:36.337018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.338473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0cad4-14d1-482b-93c9-11593a47bf6f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 96/185] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=5b027b887126434fa4af94940dfb61bc&role.id=29fc77f10237449d82dfef29036abe51 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbb4f-2053-46e6-b9cf-23ecf8216b44 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbb4f-2053-46e6-b9cf-23ecf8216b44 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dbb4f-2053-46e6-b9cf-23ecf8216b44 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 97/186] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:36.344864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dbb4f-2053-46e6-b9cf-23ecf8216b44 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.373910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.375423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.375777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.376175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.377652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.380214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a42b-1616-4f13-845f-38ab5d907165 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.382688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 90/187] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.384568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.384860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.385472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.407879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:36.409008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:36.471304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dbb4f-2053-46e6-b9cf-23ecf8216b44 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.488073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 91/188] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:36.488960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.524042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.524904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.525185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.525815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.526807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.528889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.529658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8cf11-ab89-437d-adfa-958292588d7a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.554850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 98/189] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:36 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:36.556216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.556561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.556773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.569702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.577388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:36.578630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:36.625374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a72478-7d95-4187-8d66-8f7b920e575c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.670861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.671548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.671839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.672115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.672880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:36.673188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.674785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.680498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 99/190] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:36.680498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.689511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 92/191] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.690546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.796550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.806898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0262c-b493-45e3-86c2-4c5711266740 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.812251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae11dc-c791-420f-b08d-b4dc3c2c9fda None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:36.853416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.854032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.854314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.854586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.855501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=033961e12b9349dbba61d19814615bd3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:36.855702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:36.856402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:36.857679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:36.857958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:36.858112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:36.858255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:36.865851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:36.868167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 100/192] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/role_assignments?user.id=033961e12b9349dbba61d19814615bd3&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=986616b48b464a8c82fe68f806417c4a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:36.869890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:36.871623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:36.882662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:36.883654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:36.884435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 93/193] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:36.885590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.023064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985ca4f-367c-4b7f-abe5-998ce5e547ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.026665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354aa4-7f2b-4e05-a5aa-16b497331d3e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.070460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.072292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.072812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.073285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.084053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.084053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.084053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.084381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.098731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:37.103794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.107487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.109052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 101/194] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.110251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:37.110409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:37.111536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:37.112353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:37.114896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 94/195] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:37.115996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.143948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.185677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.242582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65c5df-14d2-4165-85bc-7be7b582990b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.253221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873f581-709d-4735-ad72-06fd1ebe323f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.296493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.297090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.297377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.297653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.298417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:37.298727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.298824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.299535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.299816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.300090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.300185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.310901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 102/196] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.311777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.393158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.395592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.414268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.447625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dd1cb-e6ae-4c08-8240-5e8e9b63fb7f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.475288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.494892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.495590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.495873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.496145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.497495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:37.497803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.499698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.511020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 103/197] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.512655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.531865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 95/198] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:37.532821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.640253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093771f7-627d-48f4-adae-62cf6b8e6d18 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.641626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1aa6d-c48f-4ba7-aa97-f6629729fd8b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.653585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.679733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.680923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.681417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.681882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] PATH_INFO: `/v3/users/9414200cd2984db691d6df7b34aa5d65` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.683292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] RBAC: Authorizing `identity:get_user(user_id=9414200cd2984db691d6df7b34aa5d65)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.686159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.687564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.688530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.688884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.689141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 96/199] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/users/9414200cd2984db691d6df7b34aa5d65 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.689248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.690266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.690768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.692355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.700117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 104/200] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.701279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.772429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.799283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aba15b-eb5c-496f-ad93-48e5f625f3ed None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.838988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4aa1d-c7ee-4f21-8a97-77d893cc97c8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:37.845569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.846436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.846722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.847006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.848232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.850199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.853198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 97/201] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:37.859170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:37.884662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:37.885307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:37.885567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:37.885836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:37.926804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:37.971263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:37.973267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:37.996479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b12f8-c25d-414a-b310-596027f37d63 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.039840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.040552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:38.040662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.040927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.041515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.101823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 105/202] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:38.102683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:38.129733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.131193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.179582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:38.227196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78eff-5db7-444f-a17a-24bb4fd47c59 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.256227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 98/203] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:38.257423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:38.264296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.264986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.265270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.265546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.266643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7b9eee483a794e2c9aa153760a39f6ec) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:38.266955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.268457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.274425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 106/204] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:38 2022] GET /identity/v3/role_assignments?user.id=7b9eee483a794e2c9aa153760a39f6ec&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:38.275029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:38.314373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:38.431759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd73a-fc70-4aa7-a9fa-a5de07361194 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.450269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22e756-6d1c-4b8e-9230-4ab03f248a71 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.498676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.499859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.500613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.500891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.502401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.503934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.505705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 99/205] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:38 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:38.511668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:38.512033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.512926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.513211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.513485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.515622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.518368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.520125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 107/206] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:38 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:38.526266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:38.532745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:38.646857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c10a5a-2502-4344-9269-a37a92a15a98 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.662162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:38.705003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f6af-61d2-408c-9166-daeb648f91db None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:38.727963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.729078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.729577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.730057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.731738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, scope.project.id=73d8d392fba24d8eae5a7f396f622e19, user.id=9414200cd2984db691d6df7b34aa5d65) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:38.732261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.733965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454f368-fee9-40ff-af03-ae07f8036e73 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.740838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 100/207] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:38 2022] GET /identity/v3/role_assignments?user.id=9414200cd2984db691d6df7b34aa5d65&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:38.751604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:38.752293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:38.752572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:38.752847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:38.753709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7b9eee483a794e2c9aa153760a39f6ec) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:38.754021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:38.755558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4850b-5591-467f-988b-768ca17f9d62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:38.762923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 108/208] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:38 2022] GET /identity/v3/role_assignments?user.id=7b9eee483a794e2c9aa153760a39f6ec&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=986616b48b464a8c82fe68f806417c4a => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:39.093234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a70d6-ba63-4d10-bf8a-10aded662aab None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.093882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a70d6-ba63-4d10-bf8a-10aded662aab None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.094370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a70d6-ba63-4d10-bf8a-10aded662aab None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.096318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 101/209] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:39.111059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2aa9-aa31-48a2-948e-b2b3aa923186 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.111378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2aa9-aa31-48a2-948e-b2b3aa923186 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.111645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc2aa9-aa31-48a2-948e-b2b3aa923186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.130309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d5cc2aa9-aa31-48a2-948e-b2b3aa923186 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:39.131649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d5cc2aa9-aa31-48a2-948e-b2b3aa923186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:39.211698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 109/210] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:39.235998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a70d6-ba63-4d10-bf8a-10aded662aab None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.252655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b4dc-41bd-4908-a52d-ecf623f68da5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.252942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b4dc-41bd-4908-a52d-ecf623f68da5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.253217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b4dc-41bd-4908-a52d-ecf623f68da5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.254781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 110/211] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:39.259877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.260167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.260446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.286580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:39.287788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:39.355259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:39.379649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 111/212] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:39.384525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a70d6-ba63-4d10-bf8a-10aded662aab None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.404940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.414144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:39.428262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.429478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.429962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.430452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.442147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:39.451842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:39.452753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:39.453552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:39.456189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 102/213] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:39.462931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.532266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9f32c-d8ad-440f-8a46-94ebca252a7a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.574883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d97-cb25-43d4-95f1-300013b702cd None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.576339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.578289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.578791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.579285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.580673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:39.582326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:39.585104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 112/214] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:39.591160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.624386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.625908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.626512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.626999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.627965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:39.628723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:39.631154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:39.638200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:39.647382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 103/215] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:39.650083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.680403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:39.710631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a55f7-d52d-462d-b054-85b943a24a5a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.749657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.750707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.751173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.751663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.752588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:39.754444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:39.767342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14115615-1ca1-4dba-a93f-ddddcab12171 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.787693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 113/216] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:39 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:04:39.804645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.805677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.806003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.806351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] PATH_INFO: `/v3/users/585944759d5b4efabcf5fb5a0b9471e4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:39.822361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] RBAC: Authorizing `identity:get_user(user_id=585944759d5b4efabcf5fb5a0b9471e4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:39.823903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:39.826086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 104/217] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:39 2022] GET /identity/v3/users/585944759d5b4efabcf5fb5a0b9471e4 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:39.831710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:39.955024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57929310-dc63-47ab-b1eb-5ba97a2fd4a7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:39.988001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:39.995617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:39.997210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:39.997763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:39.998281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:40.077095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:40.078588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:40.187500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 114/218] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:40.192986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:40.258480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:40.303406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e1e7a-6722-4d63-9906-315084fe48c7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:40.339619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:40.340893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:40.341400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:40.341891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:40.343349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:40.345011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb4253-58fb-40a5-be19-0eefc6954167 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:40.347330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 105/219] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:40 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:40.353209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:40.394386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:40.477886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aefec-0d93-4856-b200-cd022e2dea26 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:40.526829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:40.527469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:40.527804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:40.528211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:40.529296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=585944759d5b4efabcf5fb5a0b9471e4) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:40.529613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:40.531125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110d433-0ee8-4a61-87a4-ebc7ba3821df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:40.537501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 115/220] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:40 2022] GET /identity/v3/role_assignments?user.id=585944759d5b4efabcf5fb5a0b9471e4&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=29fc77f10237449d82dfef29036abe51 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:40.995145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9618-ce3b-4291-af4b-b2c779a99511 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:40.995677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9618-ce3b-4291-af4b-b2c779a99511 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:40.995751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a9618-ce3b-4291-af4b-b2c779a99511 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:40.997457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 106/221] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:41.002487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033514d-b032-4797-ab74-2d3e58ee5484 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.002899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033514d-b032-4797-ab74-2d3e58ee5484 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.003064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033514d-b032-4797-ab74-2d3e58ee5484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.024188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f033514d-b032-4797-ab74-2d3e58ee5484 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:41.025409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f033514d-b032-4797-ab74-2d3e58ee5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:41.117588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 116/222] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:41.151984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a9618-ce3b-4291-af4b-b2c779a99511 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.286017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a9618-ce3b-4291-af4b-b2c779a99511 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:41.292118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87a5ad-40ac-471f-916a-76bdb5110b1c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.292409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87a5ad-40ac-471f-916a-76bdb5110b1c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.292689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87a5ad-40ac-471f-916a-76bdb5110b1c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.294867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 117/223] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:41.311103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.311430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.311711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.334511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:41.335734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:41.342659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:41.356648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:41.357458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.357746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.358028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.375892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:41.380941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:41.396189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:41.397215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:41.397986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:41.404567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 107/224] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:41.405312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.474989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 118/225] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:41.508031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.537428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f05e-6dc1-4d1d-876d-7706861f1191 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:41.595264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:41.595264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.595264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.595264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.595660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:41.595660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:41.595660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:41.611430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 108/226] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:41.614851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.699844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:41.746325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3597c7-74a7-482a-8e8c-0bd797d60101 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:41.756051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08102db-7e29-4a9a-8d1c-83ac7345e534 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:41.809547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:41.810349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.810633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.810909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.812536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:41.813981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:41.815630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 109/227] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:41.817610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:41.818414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:41.818746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:41.819023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:41.821541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.841511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:41.849659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:41.850498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:41.851300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:41.853321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 119/228] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:41.860956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:41.890606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:42.000030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b041ad-b8e1-48cb-bdaa-869f69e50212 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.031628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:42.040852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d03a4-f78d-4c7a-91ee-f1fe252e78f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.065154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.065944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.066226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.066502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.067803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.070552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.073114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 110/229] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:42.079293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:42.108185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.108906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.109197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.109472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.110227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:42.110532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.111959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.128153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 120/230] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:42.133882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:42.266328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93f9b9-c0b6-44e8-9839-175cd6aab1d9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.286176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:42.330461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c681b-3f30-4019-922a-a7da14d5a72c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.339739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.342015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.342524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.343014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/users/7b9eee483a794e2c9aa153760a39f6ec/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.345268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, user_id=7b9eee483a794e2c9aa153760a39f6ec, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.348633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.374795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 111/231] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:42 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/users/7b9eee483a794e2c9aa153760a39f6ec/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:04:42.389292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.390572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.391364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.391836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.393440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.395339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.397770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 121/232] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:42 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:42.404815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:42.471438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:42.558488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52212-ecb7-42f0-a3e3-6348f7d2287a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.620223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.621019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.621531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.621850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.624175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.625931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee5354-6de3-44f8-a89c-1b0a9fc2ce5b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.628321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 112/233] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:42 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:42.634798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:42.764600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479b1f-eec0-4cda-8a23-0e0bf024e41e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.764895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479b1f-eec0-4cda-8a23-0e0bf024e41e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.765177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61479b1f-eec0-4cda-8a23-0e0bf024e41e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.766795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 113/234] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:42.771723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0465bf-98d2-4a81-8aef-dc6088fd2f10 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.772012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0465bf-98d2-4a81-8aef-dc6088fd2f10 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.772296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0465bf-98d2-4a81-8aef-dc6088fd2f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.800188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307299-ea8a-41c7-96b4-9d4ea5dc4c6a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:42.802007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bc0465bf-98d2-4a81-8aef-dc6088fd2f10 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:42.803255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc0465bf-98d2-4a81-8aef-dc6088fd2f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:42.861367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:42.862263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:42.862543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:42.862815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/users/7b9eee483a794e2c9aa153760a39f6ec/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:42.865095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, user_id=7b9eee483a794e2c9aa153760a39f6ec, role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:42.866554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:42.892380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 122/235] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:42 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/users/7b9eee483a794e2c9aa153760a39f6ec/roles/986616b48b464a8c82fe68f806417c4a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:42.916310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 114/236] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:42.941270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:42.945556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.021840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00284d1-7e37-4103-9feb-1d2258775a99 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.022350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00284d1-7e37-4103-9feb-1d2258775a99 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.022524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00284d1-7e37-4103-9feb-1d2258775a99 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.024123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 115/237] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:43.031506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.031794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.032069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.052468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:43.054177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:43.079524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff026df-7a19-46db-bc9b-5bf9ed095f1b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.120398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.162100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.162828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.163111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.163469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.166027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 116/238] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:43.177803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:43.185745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:43.186612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:43.187468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 123/239] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:43.193005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.202823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.302796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.306611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8db1f3-cb61-4284-ac56-22c9abf0c915 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.346018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.346754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.346976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.347270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.348139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:43.348508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:43.349839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd224b6-76dd-439a-bb2f-76deae77f185 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:43.353657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66130378-a5d7-43d2-b67d-87d30a528f32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.359551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 117/240] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:43.360951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5669c-c09b-40d8-9530-51b6112511eb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.361240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5669c-c09b-40d8-9530-51b6112511eb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.361520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5669c-c09b-40d8-9530-51b6112511eb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.363015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 118/241] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:43.367894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5669c-c09b-40d8-9530-51b6112511eb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.393046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.394172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.394648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.395117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.396650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:43.398362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc89dbb-8327-459c-80e3-0e2e2cbf717f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:43.400543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 124/242] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:43.406814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.407526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.408089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.419701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.425952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:43.427404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:43.478619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5669c-c09b-40d8-9530-51b6112511eb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.505809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 125/243] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:43.506988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.529072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.529960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.530243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.530524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] PATH_INFO: `/v3/users/7ea84d619daa4a78aabbd0c25a810d3a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.552383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] RBAC: Authorizing `identity:get_user(user_id=7ea84d619daa4a78aabbd0c25a810d3a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:43.553866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:43.555638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 119/244] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/users/7ea84d619daa4a78aabbd0c25a810d3a => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:43.556475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.575715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.618320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b68c4-5825-4dab-9b6c-a3e1d1b97b20 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.650995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.659719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.660725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.660897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dadeac-b817-4d92-adf1-6da1d94e7d60 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.661360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.661842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.662855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:43.664663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:43.691943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 126/245] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:43 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:43.697007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.717887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.718815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.719106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.719418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.737583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:43.747198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:43.748173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:43.749049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 120/246] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:43.754695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:43.889068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ee860-0c7d-4809-85e1-c754ee1319f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:43.927626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:43.955296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:43.955876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:43.956161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:43.956488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:43.958796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a959398-1783-4a88-889a-b7da5310fd90 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.000642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.001291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.001620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.001936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.003438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:44.004179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.006056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.018140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 121/247] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:44.023879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:44.043507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:44.056541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.058581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.159080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1810f-7203-4286-96c7-e6915a798080 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.193940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 127/248] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:44.203211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:44.205014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.205350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.205719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.206323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] PATH_INFO: `/v3/users/585944759d5b4efabcf5fb5a0b9471e4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.207837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] RBAC: Authorizing `identity:get_user(user_id=585944759d5b4efabcf5fb5a0b9471e4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.209398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.211312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 122/249] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:44 2022] GET /identity/v3/users/585944759d5b4efabcf5fb5a0b9471e4 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:44.217587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:44.222300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:44.338458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c98e-a1df-49ab-83c6-6eb33cb2a74d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.339308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:44.376029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291e46-4b8e-46d4-900a-c2e848b5a98b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.391353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.392224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.392820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.392950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.394070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.397309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.399451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 128/250] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:44 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:44.407175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:44.431496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.433308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.433308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.433308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.435174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.436674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.438641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 123/251] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:44 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:44.444766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:44.529834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:44.546766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6471e5-5251-4c46-ab44-d10f7bb1723f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.592545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.593573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.594056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.594555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.595650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7ea84d619daa4a78aabbd0c25a810d3a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:44.596171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.597826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef8c0a-32f1-4c4d-a8d8-faf9a19d9ac4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.604402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 129/252] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:44 2022] GET /identity/v3/role_assignments?user.id=7ea84d619daa4a78aabbd0c25a810d3a&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:44.613794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3d59a-e6d1-44b2-a6fa-d16efa80d123 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:44.672812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:44.673738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:44.674062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:44.674365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/users/585944759d5b4efabcf5fb5a0b9471e4/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:44.680622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, user_id=585944759d5b4efabcf5fb5a0b9471e4, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:44.682115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ea105-cc4a-42ab-a97b-cd6f3ac1718b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:44.712893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 124/253] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:44 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/users/585944759d5b4efabcf5fb5a0b9471e4/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:45.083100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c370-b9ec-4532-9288-9224857cd154 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.083488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c370-b9ec-4532-9288-9224857cd154 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.083695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c370-b9ec-4532-9288-9224857cd154 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:45.085427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 130/254] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:45.092238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.092697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.092697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:45.114908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:45.116199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:45.124346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:45.211835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 125/255] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:45.243928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c370-b9ec-4532-9288-9224857cd154 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:45.371612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c370-b9ec-4532-9288-9224857cd154 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:45.409576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:45.410374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.410654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.410927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:45.422281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:45.430033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:45.430907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:45.431715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 131/256] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:45.437692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:45.541005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12d1ce-5729-4d1f-a7df-431c4c5ae253 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:45.576527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:45.577201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.577481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.577755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:45.578562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:45.578869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:45.580275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:45.589024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 126/257] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:45.594615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:45.608671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:45.701328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c0f1b-91fe-41e4-8ed4-1459b2a79b19 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:45.742610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:45.743513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.743795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.744070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:45.745441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:45.747675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:45.749327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 132/258] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:45 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:45.754747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:45.757660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:45.897145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63435e26-87a2-49b5-b48d-b8141340866d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:45.944512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:45.945175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:45.945464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:45.945741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.029715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:46.036488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18387cb3-a8bb-490a-8d1c-74e03a052c74 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:46.038616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:46.172525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 127/259] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:46.177821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:46.243557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4719-2a07-4c8c-95da-7b18ec461292 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.243841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4719-2a07-4c8c-95da-7b18ec461292 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.244310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4719-2a07-4c8c-95da-7b18ec461292 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.248034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 128/260] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:46.254667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.255187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.255699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.287704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:46.299211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:46.300451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:46.307010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d91f4-028d-4a8e-8a98-8420bd077789 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:46.341741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:46.342610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.342892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.343166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.344460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:46.346085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:46.348115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 133/261] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:46 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:46.353660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:46.456621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 129/262] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:46.459371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827aca-ff13-49c7-9967-35bf5bb505c8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:46.479561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:46.499869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:46.500759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.501169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.501570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.502526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7b9eee483a794e2c9aa153760a39f6ec) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:46.502998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:46.504599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:46.511159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 134/263] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:46 2022] GET /identity/v3/role_assignments?user.id=7b9eee483a794e2c9aa153760a39f6ec&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:46.559849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:46.633572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982cf0f-8075-4f6a-b072-522b514f2ae6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:46.679938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:46.681058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.681518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.681964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.700255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:46.708701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:46.709553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:46.710362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:46.712361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 130/264] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:46.719119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:46.748955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:46.833330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c038a05-a12e-4b83-85d6-7cf73264415c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:46.871544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:46.872308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:46.872610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:46.872888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:46.873652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:46.873959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:46.875394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:46.885225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 135/265] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:46.891692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:46.991462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.005532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45efa5-616d-45b3-8bd1-c326d4d11779 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:47.042288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:47.044386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.044888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.045403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] PATH_INFO: `/v3/users/7b9eee483a794e2c9aa153760a39f6ec` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.046994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7b9eee483a794e2c9aa153760a39f6ec)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:47.048732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e90416-86a6-4120-a0e7-3f777fa537d3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:47.051032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 131/266] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:46 2022] GET /identity/v3/users/7b9eee483a794e2c9aa153760a39f6ec => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:47.056945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.142324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.179052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69965e-3e79-477f-8a59-6f76c95834c2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:47.216289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:47.217389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.217865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.218332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.263625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2205d-47d3-4c5f-aee4-ff66556d4e10 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.264629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2205d-47d3-4c5f-aee4-ff66556d4e10 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.265101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2205d-47d3-4c5f-aee4-ff66556d4e10 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.267307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 132/267] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:47.279690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.280285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.280761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.298185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:47.299700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:47.312024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:47.313517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:47.372506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 133/268] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:47.395815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.468241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.513338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 136/269] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:47.519578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.577051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.598028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf422b9-4216-42c0-a762-515ab2c71663 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:47.652963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:47.654003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.654957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.655430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.670382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:47.680418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314fd75-350d-42e2-8033-d37c8b52d3e0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-21259847-fd80-4601-a32b-d45ee9280884 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:47.681195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:47.682257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:47.683085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 134/270] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:47.684371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66368-80a6-4f69-9f7b-a3ae531929ea None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.684883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66368-80a6-4f69-9f7b-a3ae531929ea None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.685311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e66368-80a6-4f69-9f7b-a3ae531929ea None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.688138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 135/271] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:47.689105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e66368-80a6-4f69-9f7b-a3ae531929ea None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.726054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:47.727433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.727923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.728398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.729856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:47.732472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226dde4-75f0-4fb1-a8d3-6f07797e209a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:47.734796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 137/272] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:47.736244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.736737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.737237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.754747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:47.756324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:47.803440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.812876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e66368-80a6-4f69-9f7b-a3ae531929ea None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:47.858550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 138/273] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:47.859542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.870110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:47.871507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:47.871920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:47.872331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:47.873323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:47.873702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:47.875517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:47.897224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 136/274] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:47.902668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:47.919801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:47.991307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a808d69-ff45-4a9c-9392-a56c53f6df38 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.016788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69e18b-8521-4870-a7f0-4997beca11fa None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.028820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.029736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.030166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.030583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.031821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7b9eee483a794e2c9aa153760a39f6ec) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:48.032306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.034140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.041998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 139/275] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/role_assignments?user.id=7b9eee483a794e2c9aa153760a39f6ec&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=986616b48b464a8c82fe68f806417c4a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:48.044080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.057515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.058917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.059564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.060134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.061791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.063764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.066348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 137/276] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:48.072294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.085984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.164113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397966f-b50f-4777-9de4-e6a6d2f2f020 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.167502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.190832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1ce69-8a39-4165-bd4b-f10e9197c4c6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.204476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.205666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.206094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.206514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] PATH_INFO: `/v3/users/05d9afcadb9746318b34c2cefd712ccf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.222948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] RBAC: Authorizing `identity:get_user(user_id=05d9afcadb9746318b34c2cefd712ccf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.224933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.227025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 140/277] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/users/05d9afcadb9746318b34c2cefd712ccf => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:48.230451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.231137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.231450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.231726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.232541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.233211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.234370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59783e1-862d-4af0-a311-328da66efcc1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.260050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 138/278] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:04:48 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:04:48.262638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6b08-3572-40ee-b639-ca35271adfe8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.263264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6b08-3572-40ee-b639-ca35271adfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.263701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6b08-3572-40ee-b639-ca35271adfe8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.267178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 139/279] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:48.268791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c18f-ba80-4494-905a-64afea4d71e9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.269224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c18f-ba80-4494-905a-64afea4d71e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.269679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849c18f-ba80-4494-905a-64afea4d71e9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.271293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 140/280] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:48.276869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802fa5-a6eb-4133-8772-2bdf0491be9e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.277758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802fa5-a6eb-4133-8772-2bdf0491be9e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.278060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802fa5-a6eb-4133-8772-2bdf0491be9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.297003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-06802fa5-a6eb-4133-8772-2bdf0491be9e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:48.298582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-06802fa5-a6eb-4133-8772-2bdf0491be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:48.378461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.387701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 141/281] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:48.390071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.390359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.390631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.395389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a592e2-8df4-4082-9ebf-f9e2037a7d6b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.408222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:48.409811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:48.443309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.443932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.444527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.444633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.445497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, user.id=05d9afcadb9746318b34c2cefd712ccf, scope.system=all) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:48.445812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.448259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.455277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 141/282] 10.222.0.39 () {56 vars in 1308 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/role_assignments?user.id=05d9afcadb9746318b34c2cefd712ccf&scope.system=all&role.id=29fc77f10237449d82dfef29036abe51 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:48.455277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.503462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.516146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 142/283] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:48.542481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.583864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c0474-837f-4248-b6ef-a2d49eb39c6c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.631642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.632455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.632788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.633075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.649683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:48.658806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:48.659799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:48.660587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 142/284] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:48.667480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.677896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a00c-ba6c-4249-91b5-c9500205d0d7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.694801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.724771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.725885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.726385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.726667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.745176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:48.755299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:48.756224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:48.757008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 143/285] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:48.764926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.817993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae747-a119-4bb9-a7d5-90f05ea3b8bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.875625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:48.890710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.892371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.892654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.892931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.893701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:48.894009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.895424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:48.912854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 143/286] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:48.920413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:48.944445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde306fd-00ea-4b94-b1eb-4f622941abe6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:48.988773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:48.989366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:48.989648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:48.989920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:48.990670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:48.990976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:48.992405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.021323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 144/287] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:49.030269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:49.094277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:49.140591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade6c99-6c33-467e-881d-c0066ed0b70c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:49.189545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:49.190423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:49.190704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:49.190974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] PATH_INFO: `/v3/users/7ea84d619daa4a78aabbd0c25a810d3a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:49.195132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] RBAC: Authorizing `identity:get_user(user_id=7ea84d619daa4a78aabbd0c25a810d3a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:49.196622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.201429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 144/288] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:48 2022] GET /identity/v3/users/7ea84d619daa4a78aabbd0c25a810d3a => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:49.204516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:49.222263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86809fa-c457-430b-80cd-69682b6969c5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:49.257949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:49.290529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:49.291623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:49.291904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:49.292176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] PATH_INFO: `/v3/users/585944759d5b4efabcf5fb5a0b9471e4` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:49.295648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] RBAC: Authorizing `identity:get_user(user_id=585944759d5b4efabcf5fb5a0b9471e4)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:49.297109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.300074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 145/289] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:49 2022] GET /identity/v3/users/585944759d5b4efabcf5fb5a0b9471e4 => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:49.308177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:49.385425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861045b-74f2-4b43-8280-e490f681de50 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:49.463615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:49.464476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:49.465617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:49.466100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:49.472025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:49.473501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.475592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 145/290] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:49 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:49.481935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:49.501906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bcc9d-0b79-4c6e-bf21-0fb1679b87b2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:49.513873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:49.565536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:49.566611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:49.567189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:49.567684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:49.641210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:49.668122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:49.670687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.679413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f42e90-a9f1-4c1c-b8a5-976026f15d21 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:49.722575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:49.724072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:49.724553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:49.725074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/users/7ea84d619daa4a78aabbd0c25a810d3a/roles/3c73ee9c7315428098a4e3ef35aad1ea` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:49.727409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, user_id=7ea84d619daa4a78aabbd0c25a810d3a, role_id=3c73ee9c7315428098a4e3ef35aad1ea)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:49.729382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:49.756090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 146/291] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:04:49 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/users/7ea84d619daa4a78aabbd0c25a810d3a/roles/3c73ee9c7315428098a4e3ef35aad1ea => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:04:49.829421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 146/292] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:49.835358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:49.943184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:49.974874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebc66e-3280-4329-b033-0cf6c1146974 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:50.022710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:50.023898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:50.024353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:50.024822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:50.026191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:50.028443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e4d27-cb34-4b2b-b23d-6c4dd8232401 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:50.030318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 147/293] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:49 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:50.036236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:50.143868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:50.163963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3b94e-dbd2-42e0-a580-6f8265b066ef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:50.198360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:50.199825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:50.200277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:50.200746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:50.201738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=585944759d5b4efabcf5fb5a0b9471e4) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:50.202220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:50.203856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b6c67-ea39-4993-b188-2e051559911b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:50.210165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 147/294] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:50 2022] GET /identity/v3/role_assignments?user.id=585944759d5b4efabcf5fb5a0b9471e4&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=29fc77f10237449d82dfef29036abe51 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:51.087323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3dbb-30a4-4783-a3e2-d948cbd3732b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.088330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3dbb-30a4-4783-a3e2-d948cbd3732b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.088823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f3dbb-30a4-4783-a3e2-d948cbd3732b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.091301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 148/295] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:51.096740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf82c-f263-4bda-9aa8-9f964782035f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.097544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf82c-f263-4bda-9aa8-9f964782035f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.098019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf82c-f263-4bda-9aa8-9f964782035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.109573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c1b20-c098-4885-a5d8-a9580e5aec6a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.110279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c1b20-c098-4885-a5d8-a9580e5aec6a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.110787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c1b20-c098-4885-a5d8-a9580e5aec6a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.112584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 149/296] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:51.118092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.118375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.118614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bf7bf82c-f263-4bda-9aa8-9f964782035f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:51.118717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.120137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bf7bf82c-f263-4bda-9aa8-9f964782035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:51.139188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:51.140995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:51.207300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 148/297] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:51.207300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29f03c-7c6a-4767-af17-38496b41d03a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.207300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29f03c-7c6a-4767-af17-38496b41d03a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.207300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29f03c-7c6a-4767-af17-38496b41d03a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.208483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 149/298] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:51.213783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.214222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.215408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.227577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 150/299] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 26 03:04:51.232184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.233064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:51.234230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:51.304888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.311312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 150/300] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:51.311312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.327513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.338876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd719f-7121-446e-8cf8-685a6d6c6eca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.385055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.386032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.387127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.387842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.389113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:51.390594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:51.396907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 151/301] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:51.396907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.422536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647f868-ba58-41bc-bff2-16329cccf2d7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.459300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.460092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.460376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.460637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.471414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:51.478915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:51.479833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:51.481304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 151/302] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:51.482847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.508457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6adcd-9d54-47c5-aa87-c6664d72366d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.542452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f357942-0207-4da4-b893-056debc67177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.543793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357942-0207-4da4-b893-056debc67177 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.544067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357942-0207-4da4-b893-056debc67177 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.544339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f357942-0207-4da4-b893-056debc67177 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.551854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.554825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f357942-0207-4da4-b893-056debc67177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:51.557600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4f357942-0207-4da4-b893-056debc67177 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:51.564610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:51.565521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 152/303] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:51.567825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f357942-0207-4da4-b893-056debc67177 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.602919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25db500-13d5-4f43-88b4-06348784808e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.640337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.641258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.641541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.641813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] PATH_INFO: `/v3/groups` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.642494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:51.644323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:51.659280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 152/304] 10.222.0.39 () {60 vars in 1134 bytes} [Thu May 26 03:04:51 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:51.668952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.680773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f357942-0207-4da4-b893-056debc67177 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.725664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.727063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.727063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.727063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.727809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:51.728119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:51.729696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:51.745131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 153/305] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:51.748130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.818456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.826798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b5bff-5f8b-4f7b-aa40-942364ee10f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.867815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:51.870099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a9-66b2-4975-9ab1-c2dfbf56f2f6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:51.876593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.877305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.877592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.877877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.878994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:51.880100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:51.882200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:51.894005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 153/306] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:51.899380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:51.915423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:51.916972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:51.916972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:51.916972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] PATH_INFO: `/v3/users/05d9afcadb9746318b34c2cefd712ccf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:51.918248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9afcadb9746318b34c2cefd712ccf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:51.919918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:51.921702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 154/307] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/users/05d9afcadb9746318b34c2cefd712ccf => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:51.926863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:52.031923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ab97-297f-403a-a6ee-0e1269db2f67 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.037642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc4ec-68dd-44c6-98c0-97ccc69ab44a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.071434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:52.073486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.074516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.074951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.075347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] PATH_INFO: `/v3/users/51e75deb99c5425e89da59cb85061185` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.080090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.080961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.081246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.081527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] PATH_INFO: `/v3/system/users/05d9afcadb9746318b34c2cefd712ccf/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.083024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=05d9afcadb9746318b34c2cefd712ccf, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:52.084900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429a575-4e9c-4ab7-9bba-f93e4ed65ad5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:52.090190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] RBAC: Authorizing `identity:get_user(user_id=51e75deb99c5425e89da59cb85061185)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:52.092034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:52.094091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 154/308] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:51 2022] GET /identity/v3/users/51e75deb99c5425e89da59cb85061185 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:52.100510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:52.101080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 155/309] 10.222.0.39 () {58 vars in 1334 bytes} [Thu May 26 03:04:51 2022] PUT /identity/v3/system/users/05d9afcadb9746318b34c2cefd712ccf/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:04:52.165468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:52.220081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f2855-a25f-4463-a07e-40c897e95639 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.258240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.258908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.259192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.259492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.260341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, user.id=51e75deb99c5425e89da59cb85061185, scope.system=all) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:52.260654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:52.262993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce544c7d-91bb-462f-92df-1a040c324bc5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:52.269369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 155/310] 10.222.0.39 () {56 vars in 1308 bytes} [Thu May 26 03:04:52 2022] GET /identity/v3/role_assignments?user.id=51e75deb99c5425e89da59cb85061185&scope.system=all&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:52.352424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba51e-0fd0-43d2-b5c1-b8c599192790 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.353077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba51e-0fd0-43d2-b5c1-b8c599192790 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.353554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ba51e-0fd0-43d2-b5c1-b8c599192790 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.356390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 156/311] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:52.362457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.362750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.363025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.380987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:52.382208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:52.462075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 156/312] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:52.485768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ba51e-0fd0-43d2-b5c1-b8c599192790 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:52.593020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ba51e-0fd0-43d2-b5c1-b8c599192790 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.629164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.629965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.630355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.630527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.641661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:52.649494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:52.650350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:52.651126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 157/313] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:52.657212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:52.707601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:52.759002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ed7a0-ffb5-4be1-8aa8-b44e369f341b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.795951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.796622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.796902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.797179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.797932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:52.798242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:52.799649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:52.809552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 157/314] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:52.814766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:52.873448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:52.926919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951f73b-5edd-4910-97cd-afd405571937 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:52.961860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:52.962735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:52.963014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:52.963405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] PATH_INFO: `/v3/users/7ea84d619daa4a78aabbd0c25a810d3a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:52.965063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] RBAC: Authorizing `identity:get_user(user_id=7ea84d619daa4a78aabbd0c25a810d3a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:52.966584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:52.968563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 158/315] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:52 2022] GET /identity/v3/users/7ea84d619daa4a78aabbd0c25a810d3a => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:52.973964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:53.083338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc2123-ebb2-4b9a-8e33-39eb5f30aea8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:53.092365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:53.121211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:53.121844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:53.122139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:53.122417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:53.193039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:53.194517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:53.299625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 158/316] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:53.304942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:53.408469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d6e03-8d3e-41d6-9e6e-260f0ff1d69d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:53.436288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:53.448098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:53.449211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:53.449753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:53.450034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:53.451268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:53.452733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfddfbe-f096-4f21-9bd7-6135ff331bb5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:53.454754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 159/317] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:53 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:53.460339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:53.470667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:53.570307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7c7a4-952b-4a9c-b7a0-82185f8f3491 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:53.606337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:53.607410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:53.607894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:53.608367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:53.609408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, scope.project.id=1685647e66434b21ac19c01987e91b17, user.id=7ea84d619daa4a78aabbd0c25a810d3a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:53.609935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:53.611603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffefed-c5fa-47cc-ba49-a102b62b4247 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:53.618271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 159/318] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:04:53 2022] GET /identity/v3/role_assignments?user.id=7ea84d619daa4a78aabbd0c25a810d3a&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:54.058212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbbc-30e9-46d1-ad43-bc39261a8b7b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.058498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbbc-30e9-46d1-ad43-bc39261a8b7b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.058975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbbc-30e9-46d1-ad43-bc39261a8b7b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.060549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 160/319] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:54.065675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.065969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.066246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.083261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:54.084470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:54.149850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:54.156354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 160/320] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:54.179334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dbbbc-30e9-46d1-ad43-bc39261a8b7b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.285760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7dbbbc-30e9-46d1-ad43-bc39261a8b7b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:54.319409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:54.320123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.320400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.320676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.321835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:54.323298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff19df8-2063-4fa7-8ffc-018ecd7f0883 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:54.325445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 161/321] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:54.331009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.376274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce3c5-3539-4a86-8734-c27f5a0b8e79 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.376567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce3c5-3539-4a86-8734-c27f5a0b8e79 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.376847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce3c5-3539-4a86-8734-c27f5a0b8e79 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.378398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 162/322] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:04:54.383226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.383541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.383814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.402345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:54.403738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:54.447559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c3a5-3988-4186-9a75-494e1ac908ea None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:54.468477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:54.484880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:54.485078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 163/323] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:54.485317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.485585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.485853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] PATH_INFO: `/v3/groups` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.486539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:54.488000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce15688-5a72-47ef-acc2-e49ad7edb831 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:54.504911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 161/324] 10.222.0.39 () {60 vars in 1134 bytes} [Thu May 26 03:04:54 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:04:54.513032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.611745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9950-0bf6-441e-8ff1-59e7a32d9f1b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.612041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9950-0bf6-441e-8ff1-59e7a32d9f1b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.612222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c9950-0bf6-441e-8ff1-59e7a32d9f1b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.613763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 162/325] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:54.619250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.619562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.619841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.626500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24f4d3-950a-4e2b-b604-357dd55a9430 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:54.637514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:54.638744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:54.666387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:54.667196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.668226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.668694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.680038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:54.689154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:54.690898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:54.691851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 164/326] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:54.695230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.708625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:54.726227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 163/327] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:54.748238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.763465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:54.831276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f18fe-b1b6-4d81-9f78-7d0315b824a0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:54.869714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508d7f-ffdb-44d4-9824-e51e6e20e406 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:54.875296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:54.876243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.876722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.877248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.878014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:54.878323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:54.879795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:54.891290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 165/328] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:54.896717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:54.919168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:54.920165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:54.920497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:54.920798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:54.932506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:54.941092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:54.942193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:54.943091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 164/329] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:54.949384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:55.014126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:55.048208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d2a5-55b9-4dbf-873b-60da98933fad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:55.067989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:55.076950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f4fd-08ae-4066-a5f8-75fddfa2c743 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:55.094428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:55.095892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:55.096791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:55.097865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] PATH_INFO: `/v3/users/05d9afcadb9746318b34c2cefd712ccf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:55.099470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] RBAC: Authorizing `identity:get_user(user_id=05d9afcadb9746318b34c2cefd712ccf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:55.101449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:55.105352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 166/330] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/users/05d9afcadb9746318b34c2cefd712ccf => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:55.110811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:55.117350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:55.118030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:55.118314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:55.118594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:55.119393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:55.119705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:55.121692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:55.131899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 165/331] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:55.137901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:55.248470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e498-d31d-464a-bea8-66872f1c0af9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:55.266088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbdcf0-6d6b-47a7-8b52-2bc69b31334d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:55.283985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:55.289298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:55.290567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:55.291110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:55.291593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:55.293307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] RBAC: Adding query filter params (role.id=29fc77f10237449d82dfef29036abe51, user.id=05d9afcadb9746318b34c2cefd712ccf, scope.system=all) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:55.293812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:55.295768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:55.311022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 167/332] 10.222.0.39 () {56 vars in 1308 bytes} [Thu May 26 03:04:55 2022] GET /identity/v3/role_assignments?user.id=05d9afcadb9746318b34c2cefd712ccf&scope.system=all&role.id=29fc77f10237449d82dfef29036abe51 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:55.314707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:55.315885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:55.316358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:55.316841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] PATH_INFO: `/v3/users/51e75deb99c5425e89da59cb85061185` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:55.318388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] RBAC: Authorizing `identity:get_user(user_id=51e75deb99c5425e89da59cb85061185)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:55.320142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22235a-65bf-426f-a0e7-715004c061fb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:55.322423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 166/333] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:04:55 2022] GET /identity/v3/users/51e75deb99c5425e89da59cb85061185 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:04:55.328920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:55.381329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:55.462708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c108-151d-463e-af5e-4306527d2f38 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:55.506909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:55.508138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:55.508420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:55.508699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] PATH_INFO: `/v3/system/users/51e75deb99c5425e89da59cb85061185/roles/3c73ee9c7315428098a4e3ef35aad1ea` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:55.510266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=51e75deb99c5425e89da59cb85061185, role_id=3c73ee9c7315428098a4e3ef35aad1ea)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:55.512559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce9782-88a7-40cc-9453-5f68a9dc729c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:55.529872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 168/334] 10.222.0.39 () {58 vars in 1334 bytes} [Thu May 26 03:04:55 2022] PUT /identity/v3/system/users/51e75deb99c5425e89da59cb85061185/roles/3c73ee9c7315428098a4e3ef35aad1ea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:04:57.597039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1299a1-bc2c-431a-85f4-54233ad67da4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:57.597928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1299a1-bc2c-431a-85f4-54233ad67da4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:57.598353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1299a1-bc2c-431a-85f4-54233ad67da4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:57.600130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 167/335] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:57.608696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:57.609603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:57.610085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:57.630251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:57.631515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:57.728847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 169/336] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:04:57.768699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1299a1-bc2c-431a-85f4-54233ad67da4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:57.895004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1299a1-bc2c-431a-85f4-54233ad67da4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:57.940475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:57.941306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:57.941585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:57.941857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:57.958493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-056c7e2e-ce69-4ecf-b8ef-850723aa2112 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:57.967613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:57.968548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:04:57.969803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 168/337] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:04:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:57.975039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:57.999496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df492-e069-4c07-be73-1154061e52aa None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.000002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df492-e069-4c07-be73-1154061e52aa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.000490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df492-e069-4c07-be73-1154061e52aa None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.003158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 169/338] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:58.015274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.016041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.016568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.025077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.045202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:58.046394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:58.139454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.162150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353476-4d3f-4a02-9b73-348cb3085cbc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:58.206596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 170/339] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:58.207492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:58.208317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.208760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.209177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.210073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:58.210520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:58.212071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:58.221297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 170/340] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:04:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:58.226919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:58.242029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:58.359610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e26f-a3f5-4c00-927b-9ca586a6b844 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:58.399968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:58.400969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.401460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.401997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.430455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.433741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dcbcf-1484-4006-a438-7fb6909cfd71 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:58.456236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.499022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:58.500926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.501908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.504651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.513811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:58.515522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72844583-a3b6-4ee0-9f56-4893131b74d5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:58.529223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-aa6a5156-6b96-4cf7-b3a4-90561840a906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:58.537417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:58.538488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:58.539457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:58.549086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 171/341] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:04:58.551707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87727462-8fe2-45b7-aecd-b71e3005e080 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.553477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87727462-8fe2-45b7-aecd-b71e3005e080 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.554754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87727462-8fe2-45b7-aecd-b71e3005e080 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.559654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 172/342] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:58.560705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87727462-8fe2-45b7-aecd-b71e3005e080 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:58.697400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 171/343] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:58.698711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.699210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.699787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.713709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87727462-8fe2-45b7-aecd-b71e3005e080 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:58.740020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:58.741212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:58.792039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:58.792880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:58.793163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:58.793610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:58.794516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:58.794968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:58.796588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:58.809004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 173/344] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:58.810321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:58.835948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.853304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:58.862967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 172/345] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:58.863816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.035586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e6da6-0ddb-4d87-9192-6fcef91fe8d4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.041205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e66c-5a1f-41d9-b5fc-5b0f44ab99a6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.075279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.076299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.076765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.077231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.094808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.096100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.096595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.097060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.098701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.100542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.102990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 174/346] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:58 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.104145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.171998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.179844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.181737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cf196-72fd-4797-a831-21be8583308b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.237906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.269322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32094402-af45-4ec8-900b-60826f07af2e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.312008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.312463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 173/347] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:04:59.312728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.313003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.313269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.318143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06132ce-711f-42f8-bfff-37600128496b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.318434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06132ce-711f-42f8-bfff-37600128496b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.318715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06132ce-711f-42f8-bfff-37600128496b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.320394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 174/348] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:59.322416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06132ce-711f-42f8-bfff-37600128496b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.329768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-36a9f9b8-98a4-43c7-982a-ada4e783b85e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:04:59.337499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:04:59.338456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:04:59.339829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 175/349] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:04:59.340973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125abfc-7239-4e65-ba3d-0bfe7b3d8b97 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.340973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125abfc-7239-4e65-ba3d-0bfe7b3d8b97 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.341249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125abfc-7239-4e65-ba3d-0bfe7b3d8b97 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.343365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 176/350] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:59.344361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125abfc-7239-4e65-ba3d-0bfe7b3d8b97 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.450806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.458366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06132ce-711f-42f8-bfff-37600128496b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.486067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125abfc-7239-4e65-ba3d-0bfe7b3d8b97 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.499360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.500301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.500498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.500768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.508121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.509583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb6cd-d29f-47df-90f8-73e2cd0b98cd None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.511585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 175/351] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.513091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70263-0fc9-4a98-b88b-fe238cba4b2d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.513374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70263-0fc9-4a98-b88b-fe238cba4b2d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.513748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70263-0fc9-4a98-b88b-fe238cba4b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.526020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.527625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.529126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.529405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.530622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.531173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f1c70263-0fc9-4a98-b88b-fe238cba4b2d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:59.532112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.532404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1c70263-0fc9-4a98-b88b-fe238cba4b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:59.533964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 177/352] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.538414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.553735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.615338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 176/353] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:59.616924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.617218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.617491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.636729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:04:59.638361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:04:59.655064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d76288-6a78-4c62-9f04-0de6e709eeeb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.694256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.695274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.695752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.696233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.696992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:59.697289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.698908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b943d5f-59fa-4630-bc46-695bf3f11c07 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.707789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 178/354] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.709080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d63669-37e0-4ba5-9650-436657fb9b91 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.709368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d63669-37e0-4ba5-9650-436657fb9b91 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.709635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d63669-37e0-4ba5-9650-436657fb9b91 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.710864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 179/355] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:04:59.711567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d63669-37e0-4ba5-9650-436657fb9b91 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.719031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 177/356] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:04:59.720518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.749143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.800123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:04:59.837933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d63669-37e0-4ba5-9650-436657fb9b91 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.849740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94ce6a-7ec4-499e-8da7-29b936882e62 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:04:59.880635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.881573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.881853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.882125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.882960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=73d8d392fba24d8eae5a7f396f622e19) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:04:59.883323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.884766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.887065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:04:59.887993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:04:59.888275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:04:59.888553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:04:59.889744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:04:59.891292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 180/357] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=986616b48b464a8c82fe68f806417c4a => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.891603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:04:59.892103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:04:59.893453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 178/358] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:04:59.894238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.019867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.032172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16283f-6c0f-404a-9366-c12a1a3e96f1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.037247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8ee40-da92-4246-9874-5f340e347003 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.080926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.081597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.081953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.082157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.083598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.084405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.084688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.085290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.093454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.100676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e1cab87c-a491-473a-a507-a508d9da636e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:00.109786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:00.110724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:00.111547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 181/359] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:00.112893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.112893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.112893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.142172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:00.143850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:00.184716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:00.186519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:00.235466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 182/360] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:00.236319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.303572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.315934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 179/361] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:00.316837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.363315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71206-4e1d-4152-979e-f9c2b5b40ffa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.406960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.418959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.419844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.420124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.420396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.434784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:00.444536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:00.445640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:00.446586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 183/362] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:00.447358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.455411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65f84f-3749-42e4-bb13-bdd644415c61 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.499788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.500453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.500732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.501003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.501830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=73d8d392fba24d8eae5a7f396f622e19) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:00.502554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:00.504708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:00.511718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 180/363] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=29fc77f10237449d82dfef29036abe51 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:00.513545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.587679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8931-5ad4-4e3c-af16-01a97b702a7e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.593549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.638777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.639778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.640071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.640352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.641386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:00.641702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:00.643454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:00.653529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef11ac-c36c-4260-97c9-c1c3cc27cf88 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.658052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 184/364] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:00.658897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.699986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.703357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.704024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.704508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.722739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:00.731568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:00.732523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:00.733329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:00.733927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.735995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 181/365] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:00.737141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.814693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09233e61-45c9-48d3-b32b-3f9fdb6f4df6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.857884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.858752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.859031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.861284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.861701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:00.863150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:00.867431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 185/366] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:00.867431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:00.874609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3748b-a78b-4d35-aaad-21770b7fec0e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:00.892465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:00.915339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:00.915990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:00.916275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:00.916550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:00.918230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:00.918230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:00.919647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:00.928875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 182/367] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:00.929844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.013023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1ba34-e9c4-4045-9511-227e3e3be641 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.036451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.058306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.059947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.059947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.059947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] PATH_INFO: `/v3/users/51e75deb99c5425e89da59cb85061185` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.061710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] RBAC: Authorizing `identity:get_user(user_id=51e75deb99c5425e89da59cb85061185)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.062904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.064882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 186/368] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/users/51e75deb99c5425e89da59cb85061185 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.065084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9be1f-0a62-478b-b9db-b455576c1ee2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.065653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.109427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.109769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.110097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.110380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.111129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:01.112880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.114267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.124681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 183/369] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.125659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.179984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.200085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476ad7-2a31-4e39-a7b2-049081167d62 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.237389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.238257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.238541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.238946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.240206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.241659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.243798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 187/370] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.244577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.254793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a2653-1291-4da4-a30b-642a833f940a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.290122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.291117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.291643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.292640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.302848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.356902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa323-42e5-4b8c-83bd-f4f33bfe303f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.365188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.369013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.399624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.400945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.401224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.401530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.402685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] RBAC: Adding query filter params (role.id=3c73ee9c7315428098a4e3ef35aad1ea, user.id=51e75deb99c5425e89da59cb85061185, scope.system=all) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:01.403070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.404499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.416712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.422017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 188/371] 10.222.0.39 () {56 vars in 1308 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/role_assignments?user.id=51e75deb99c5425e89da59cb85061185&scope.system=all&role.id=3c73ee9c7315428098a4e3ef35aad1ea => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.424021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.485545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 184/372] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:01.486461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.547185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494b114-0d7d-4fce-a2d6-6bb33d4af5ca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.559981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.588059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.589679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.590215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.590630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.596062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf970-80e1-4510-ba67-9c6b07e573d6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.632210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.633046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.633428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.633807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.634762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=1685647e66434b21ac19c01987e91b17) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:01.635204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.637279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.643871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 185/373] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=29fc77f10237449d82dfef29036abe51 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.644833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.655935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.675794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.677676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.795022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 189/374] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:01.800396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.815602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:01.842588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aaf89-2e6e-4056-a4c3-6340669fb7ad None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.927483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.928419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.928621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.928917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.930079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.931219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06f484-db36-45d0-a866-476b02ca77fa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:01.931549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.939726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 186/375] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:01.968855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:01.978237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:01.980197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:01.980701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:01.981290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] PATH_INFO: `/v3/projects/fb9a8fcaeb99475786d754ce49440ef8` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:01.983571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] RBAC: Authorizing `identity:get_project(project_id=fb9a8fcaeb99475786d754ce49440ef8)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:01.985817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:01.989084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 190/376] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/projects/fb9a8fcaeb99475786d754ce49440ef8 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:01.994300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:02.063026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:02.099498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaf021-38a6-41fe-9a2f-a41fc5dfaec4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:02.107008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:02.153389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29055c-e8ba-48c0-b2d6-65d47f1fc2ff None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:02.157249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:02.158120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.158398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.158672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.160800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:02.162079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:02.163763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 187/377] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:02.170302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:02.219822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:02.221357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.221640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.221910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] PATH_INFO: `/v3/groups/2eed0f41cfcd4212ae0056233ac487e6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.233173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] RBAC: Authorizing `identity:get_group(group_id=2eed0f41cfcd4212ae0056233ac487e6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:02.234629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:02.236632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 191/378] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:01 2022] GET /identity/v3/groups/2eed0f41cfcd4212ae0056233ac487e6 => generated 229 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:02.244798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:02.337522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77e02-8026-4ccc-be72-132c7dae3516 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:02.408806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:02.409483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.409762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.410038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.410882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=1685647e66434b21ac19c01987e91b17) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:02.411190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:02.412679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d1bd6-7d36-46db-b541-f3550c23d186 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:02.427304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 188/379] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:02 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=986616b48b464a8c82fe68f806417c4a => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:02.447605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500a70b-9fbd-45a5-a0dc-2752ef8b2f3f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:02.504519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:02.505194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.505473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.505746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.506605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] RBAC: Adding query filter params (group.id=2eed0f41cfcd4212ae0056233ac487e6, role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=fb9a8fcaeb99475786d754ce49440ef8) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:02.506918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:02.508472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d441fd-cd60-4965-920b-15db218d22dd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:02.517488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 192/380] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:02 2022] GET /identity/v3/role_assignments?group.id=2eed0f41cfcd4212ae0056233ac487e6&scope.project.id=fb9a8fcaeb99475786d754ce49440ef8&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:02.703019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19a9aa-3fc5-4b22-b0e7-ffc66c13079c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.703534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19a9aa-3fc5-4b22-b0e7-ffc66c13079c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.703926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19a9aa-3fc5-4b22-b0e7-ffc66c13079c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.705676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 189/381] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:02.711162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:02.711490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:02.711748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:02.725968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:02.733443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:02.734666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:02.826857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 193/382] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:02.864445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19a9aa-3fc5-4b22-b0e7-ffc66c13079c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:02.868503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:03.000013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19a9aa-3fc5-4b22-b0e7-ffc66c13079c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.048041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.048876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.049160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.049432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.061465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cfb1c0ae-ca01-46be-9ac4-1f0107abeea0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:03.069780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:03.070667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:03.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 190/383] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:03.079078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:03.204095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc3f7-3e67-4888-8cf4-67fccf9304c9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.219639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488d50-a59e-4825-8f78-9cb9edb13b99 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.220224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488d50-a59e-4825-8f78-9cb9edb13b99 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.220680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61488d50-a59e-4825-8f78-9cb9edb13b99 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.222212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 191/384] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:03.231107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.231107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.231107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.253403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:03.254522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:03.267509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.268192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.268472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.268740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.269495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:03.269801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:03.271192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:03.279579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:03.289899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 194/385] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:03.295932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:03.392094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:03.409895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 192/386] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:03.443898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bfb67-72da-4f45-84b3-7fb5abef28c0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.460299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:03.479197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.481373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.481893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.482370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.483935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:03.485908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:03.488414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 195/387] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:03 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:03.493794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:03.563717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:03.598938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb86d054-80f4-4fbd-8775-605eacdd22a4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.634488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.635652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.636565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.636960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.638347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:03.639964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:03.642023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 196/388] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:03 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:03.647564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:03.733160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca8298-0522-4bd1-a567-613cccab9250 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.751707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a2e1f-f7b1-48b9-ad9e-854a71b1b1d6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:03.787354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.788384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.788791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.789169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/groups/91d670111e13408ab32d325a8f4e636c/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.791545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, group_id=91d670111e13408ab32d325a8f4e636c, role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:03.793690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:03.798698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:03.813518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 197/389] 10.222.0.39 () {58 vars in 1442 bytes} [Thu May 26 03:05:03 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/groups/91d670111e13408ab32d325a8f4e636c/roles/986616b48b464a8c82fe68f806417c4a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:05:03.826217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:03.827352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:03.827833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:03.828287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:03.847727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:03.857130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:03.858838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:03.860286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:03.864138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 193/390] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:03.873351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:04.030398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71eefc5-a1e0-4bd0-8720-b00f8fb13a6e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:04.069182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:04.070142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.070600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.071052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.072033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:04.072518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:04.074131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:04.083412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 198/391] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:05:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:04.088759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:04.172997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:04.232237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc801-ab65-4478-8624-128cb6dab3c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:04.286256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:04.287434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.287894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.288342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.296001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:04.297719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:04.299950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 194/392] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:04 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:04.305217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:04.393848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:04.460287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2418-a8ff-4ae1-9360-a1aebdccd0d2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:04.510730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:04.511924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.512385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.512935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.514330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:04.516248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83cd9-4971-4cb9-82ca-9f2e1b6a4906 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:04.518491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 199/393] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:04 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:04.523940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:04.543311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cdb9-b4bd-4148-aa41-c6aaf8d840da None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.543954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cdb9-b4bd-4148-aa41-c6aaf8d840da None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.545685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cdb9-b4bd-4148-aa41-c6aaf8d840da None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.547289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 200/394] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:04.560934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.561457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.561881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.593259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:04.594933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:04.696326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59602-81e2-4987-8b8e-a3efb39dbe75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:04.717951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 201/395] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:04.739616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:04.740246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:04.753195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:04.760264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.760687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.761343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19/groups/91d670111e13408ab32d325a8f4e636c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.768035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] RBAC: Authorizing `identity:create_grant(project_id=73d8d392fba24d8eae5a7f396f622e19, group_id=91d670111e13408ab32d325a8f4e636c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:04.769606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:04.810158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:04.810505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 195/396] 10.222.0.39 () {58 vars in 1442 bytes} [Thu May 26 03:05:04 2022] PUT /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19/groups/91d670111e13408ab32d325a8f4e636c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:05:04.893079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03494268-7751-4aa9-88cf-e16d9996b1f8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:04.946884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:04.948088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:04.949224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:04.949917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:04.968203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-56f00625-a68a-4c7b-a2e9-9de9ee0c2ee1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:04.976599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:04.977560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:04.978430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:04.982101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 202/397] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:04.987690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.085111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268d627-26a3-470f-a264-ea5e40d8694e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.085742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268d627-26a3-470f-a264-ea5e40d8694e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.086220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268d627-26a3-470f-a264-ea5e40d8694e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.090026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 203/398] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:05.094942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012219cd-d702-460e-9248-292f24fc2141 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.095557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012219cd-d702-460e-9248-292f24fc2141 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.096053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012219cd-d702-460e-9248-292f24fc2141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.116058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-012219cd-d702-460e-9248-292f24fc2141 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:05.117588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-012219cd-d702-460e-9248-292f24fc2141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:05.117697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1fe13-991f-4fae-bbe2-64ac68071b72 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.172313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.173239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.173667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.174083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.174978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:05.175519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.177683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.190688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 196/399] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:05:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:05.196284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.209542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 204/400] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:05.233342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012219cd-d702-460e-9248-292f24fc2141 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.284295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:05.286927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:05.347186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012219cd-d702-460e-9248-292f24fc2141 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.383707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.384739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.385167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.385597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.392078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8214b-e1e7-43ff-802e-a2ee67497ea6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.396797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:05.405812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:05.406709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:05.407540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:05.410250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 205/401] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:05.417963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.478178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.479173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.480136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.480794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.488367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.490221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.492601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 197/402] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:05.499902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.544797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d4bf3-64a3-4a6d-89ab-3cf9278d11ae None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.547708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:05.581920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.582868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.583322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.583743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.584666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:05.585120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.586647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.596221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 206/403] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:05.602155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.703146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06a945-dcdb-44f9-a105-7565e9df321a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.707992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09d46-ba0c-4df2-beae-b7ffc242da35 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.717368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:05.742357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.743443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.743591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.743907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.744332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.744566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.744847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.745191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.746484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.746642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.747998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecf394-0a0e-4a13-8ae2-4721720087a8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.748411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.749912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 207/404] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:05.750452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 198/405] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:05.751093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629080f-db14-43df-9cb8-934c0682ad62 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.755736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629080f-db14-43df-9cb8-934c0682ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.756046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629080f-db14-43df-9cb8-934c0682ad62 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.756408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.757568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 208/406] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:05.758427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629080f-db14-43df-9cb8-934c0682ad62 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.804375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:05.877474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91af96-641c-429a-9280-be24c8bd4290 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.889719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629080f-db14-43df-9cb8-934c0682ad62 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:05.916262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.918507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.918940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.919406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/groups/91d670111e13408ab32d325a8f4e636c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.921806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, group_id=91d670111e13408ab32d325a8f4e636c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.923489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.926547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:05.928135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.928526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.928860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.930199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:05.931803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9d143-1f81-42a9-893f-f9cd58b0d127 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:05.939027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 209/407] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:05 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:05.940498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:05.940858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:05.941214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:05.942403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 199/408] 10.222.0.39 () {58 vars in 1442 bytes} [Thu May 26 03:05:05 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/groups/91d670111e13408ab32d325a8f4e636c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:05.943499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:05.959150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:05.960408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:05.994228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:06.033945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 210/409] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:06.088138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d90caf-a4bb-4637-8be7-877c8e97b58b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:06.101335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:06.132116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:06.133842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.134310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.134757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17/groups/91d670111e13408ab32d325a8f4e636c/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.143393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] RBAC: Authorizing `identity:create_grant(project_id=1685647e66434b21ac19c01987e91b17, group_id=91d670111e13408ab32d325a8f4e636c, role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:06.145525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:06.167680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:06.171023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 200/410] 10.222.0.39 () {58 vars in 1442 bytes} [Thu May 26 03:05:05 2022] PUT /identity/v3/projects/1685647e66434b21ac19c01987e91b17/groups/91d670111e13408ab32d325a8f4e636c/roles/986616b48b464a8c82fe68f806417c4a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:06.271307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c9df0-1aaf-47c2-9ad4-f440f734d1c9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:06.312520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:06.313724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.314208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.314753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.327356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:06.335559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:06.336502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:06.337313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:06.340775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 211/411] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:06.346939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:06.462337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:06.466740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e4497-6cf0-4c0c-91f0-92b42f60b822 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:06.522685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:06.523744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.524234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.524728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.525715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:06.526232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:06.528925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:06.544354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 201/412] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:06.550294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:06.685101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:06.705438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cde33-5a2e-439b-827e-24aa52d10291 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:06.777386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:06.778214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.778498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.778778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] PATH_INFO: `/v3/groups/2eed0f41cfcd4212ae0056233ac487e6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.783980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] RBAC: Authorizing `identity:get_group(group_id=2eed0f41cfcd4212ae0056233ac487e6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:06.785482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1f312-f61e-4678-b593-3c2726fdab2c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:06.787525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 212/413] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:06 2022] GET /identity/v3/groups/2eed0f41cfcd4212ae0056233ac487e6 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:06.793600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:06.862397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234b50-46cb-4353-a491-612fdc756ebd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.862786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234b50-46cb-4353-a491-612fdc756ebd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.862968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36234b50-46cb-4353-a491-612fdc756ebd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.864665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 213/414] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:06.876664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.876818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.877125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.905219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:06.906424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:06.920103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be847-7c7d-48a0-a96a-def427829381 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:06.956031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:06.957279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:06.957776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:06.958245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] PATH_INFO: `/v3/projects/fb9a8fcaeb99475786d754ce49440ef8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:06.959673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] RBAC: Authorizing `identity:get_project(project_id=fb9a8fcaeb99475786d754ce49440ef8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:06.961311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:06.963644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 202/415] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:06 2022] GET /identity/v3/projects/fb9a8fcaeb99475786d754ce49440ef8 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:06.969478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:07.027714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:07.075009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140bad8-9741-40a5-bf31-8eabbf1c4a34 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:07.113067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:07.113310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 214/416] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:07.115132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.115642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.116116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] PATH_INFO: `/v3/projects/fb9a8fcaeb99475786d754ce49440ef8/groups/2eed0f41cfcd4212ae0056233ac487e6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.118351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] RBAC: Authorizing `identity:create_grant(project_id=fb9a8fcaeb99475786d754ce49440ef8, group_id=2eed0f41cfcd4212ae0056233ac487e6, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:07.120683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:07.144692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 203/417] 10.222.0.39 () {58 vars in 1442 bytes} [Thu May 26 03:05:06 2022] PUT /identity/v3/projects/fb9a8fcaeb99475786d754ce49440ef8/groups/2eed0f41cfcd4212ae0056233ac487e6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:05:07.163480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:07.287682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:07.298299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb1ef3-c811-4033-b0d5-9d058a36752b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:07.336255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:07.337065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.337343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.337611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.349095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:07.357954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:07.358954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:07.359774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 215/418] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:07.366755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:07.498672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:07.510585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bffd9-aa89-456f-9182-1fb47b5ca55c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:07.550674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:07.551296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.551589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.551867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.552625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:07.552937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:07.554333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d229d-f35d-4cae-a424-7bb138f3e70f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:07.566507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 204/419] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:07.572323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:07.716484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47044e83-792c-4e13-9390-52b86cdb9744 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:07.751474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:07.752126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.752408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.752682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.772418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:07.824316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:07.825728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:07.904874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a38f-9993-41d1-8a50-60c0adc5f969 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.905193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a38f-9993-41d1-8a50-60c0adc5f969 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.905429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a38f-9993-41d1-8a50-60c0adc5f969 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.919691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 205/420] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:07 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:07.928777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:07.928986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:07.929253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:07.931810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 216/421] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:07.946021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:07.987298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:07.987298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:08.084017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:08.156885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 206/422] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:08.172640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949cc28-47b4-4ec0-8148-af0d95db0929 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:08.189711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:08.261998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:08.262877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.263162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.263467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.272046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:08.273542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:08.278335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 217/423] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:07 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:08.285242 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:08.439802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:08.473145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93261b-271c-40d0-bc73-4f267d15f341 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:08.531276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542b29-d405-4af8-8d94-c2faebfe1d50 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:08.544780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:08.545596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.545876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.546145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.566891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:08.574965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:08.575958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:08.576788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 207/424] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:08.578318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:08.584277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:08.600573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:08.601574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.601929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.602879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.604272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:08.605941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:08.608146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 218/425] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:08 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:08.614691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:08.729902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e1c9-d4fb-45bb-9ee1-d867612c02d4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:08.771506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:08.772341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.772957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.773330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.774675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=73d8d392fba24d8eae5a7f396f622e19) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:08.775257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:08.778673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617a4bb-3d5d-455d-9607-c0e3ad1a58ab None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:08.785465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 219/426] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:08 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=986616b48b464a8c82fe68f806417c4a => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:08.788032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937b85-3afa-401b-8f8b-3b0e5556e380 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.788550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937b85-3afa-401b-8f8b-3b0e5556e380 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.789054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63937b85-3afa-401b-8f8b-3b0e5556e380 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.790595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 220/427] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:08.797857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.798173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.798400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.819879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:08.827465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:08.859792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77e048-0d29-4afd-af00-7e753a940a04 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:08.896626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:08.910643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:08.911622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.911981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.912329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.913485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:08.913869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:08.916339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe600d-06a9-4692-825d-f1b66b6c2d85 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:08.928409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 221/428] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:08.928704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 208/429] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:05:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:08.932726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0fc4-4acf-4ed2-b349-45346c8bec70 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.933653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0fc4-4acf-4ed2-b349-45346c8bec70 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.934184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0fc4-4acf-4ed2-b349-45346c8bec70 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.936234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:08.942145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 209/430] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:08.944119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:08.944521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:08.944990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:08.952005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:08.966194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:08.967584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:09.051662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0f4b8-d6dd-4d7d-a209-4e0eb0ce1ee7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.054474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 210/431] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:09.055100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.092509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.093162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.093438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.093715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.153253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.176743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.178217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.194250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc6078-4b99-4918-8e21-b82a55ea15d6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.230395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.230935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.231219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.231524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.233384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.242576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:09.250718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:09.251648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 211/432] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:09.257403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.287114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 222/433] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:09.288029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.364147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9bbdc-2153-4f5d-89bf-84c259fcd42d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.379940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.402339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5f72c-b935-44ef-80d0-6dfdb6ac04af None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.408365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.409406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.409926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.410269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.421416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:09.428943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:09.429792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:09.430660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 212/434] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:09.431353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.443905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.444567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.444842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.445302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.445904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:09.446209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.447625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.456660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 223/435] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:09.457455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.496884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.544385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c51c-52e5-446c-a7d4-e9453c48fc30 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.568186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc508b31-1642-432d-a688-ac233959e4f8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.581075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.581918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.582195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.582465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] PATH_INFO: `/v3/projects/73d8d392fba24d8eae5a7f396f622e19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.583926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] RBAC: Authorizing `identity:get_project(project_id=73d8d392fba24d8eae5a7f396f622e19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.585363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.587256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 213/436] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/projects/73d8d392fba24d8eae5a7f396f622e19 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:09.592122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.605124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.606403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.606722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.606976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.607780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] RBAC: Adding query filter params (name=member) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:09.608096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.609499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.618051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 224/437] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:09.618782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.622438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.703664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220b70-1fbe-4fa8-9cbe-2abf00a53de5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.728193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.729847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c294-5a18-4ec1-ad1b-390e40e84b0b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.741726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.742386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.742669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.742946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.765136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.766285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.766566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.766840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.768161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.769675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.771800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 225/438] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:09.772458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.827957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:09.829824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:09.857542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:09.891547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736530b3-99d1-4982-aac7-9329041caad5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:09.926753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:09.927432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:09.927715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:09.927989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:09.941362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 214/439] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:09.942172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:09.967444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.002626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.004116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705f8fd-aa5a-4660-933a-94500b3b88d5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.044010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d422517-4845-4930-911d-c9481f52d551 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.070211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.077354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.077998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.078271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.078540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.079390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=73d8d392fba24d8eae5a7f396f622e19) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:10.079696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.081104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.086741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 215/440] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=73d8d392fba24d8eae5a7f396f622e19&role.id=29fc77f10237449d82dfef29036abe51 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.087593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.103450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 226/441] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:10.106469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eef1c-4c1a-44ea-af3c-13252ad6aafe None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.107051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eef1c-4c1a-44ea-af3c-13252ad6aafe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.107517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eef1c-4c1a-44ea-af3c-13252ad6aafe None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.109230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 227/442] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:10.110197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eef1c-4c1a-44ea-af3c-13252ad6aafe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.194847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.213643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeac134-9d87-47e9-8b9f-1842fb9e1d2c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.241958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eef1c-4c1a-44ea-af3c-13252ad6aafe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.253319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.254500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.254785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.255054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.256312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.257800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7adea-47f7-4fd0-bbc7-3f9c82c508cf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.259589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 216/443] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.261330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.261606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.261878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.279935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:10.281126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:10.284230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.285090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.285372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.285644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] PATH_INFO: `/v3/projects/1685647e66434b21ac19c01987e91b17` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.286829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] RBAC: Authorizing `identity:get_project(project_id=1685647e66434b21ac19c01987e91b17)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.288302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.290153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 228/444] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/projects/1685647e66434b21ac19c01987e91b17 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.290988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.336028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.375137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 217/445] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:10.376695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.423189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4ff26-d0ed-49fb-9a7d-1daaca8a3c99 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.463364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.463731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.464524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.464808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.468183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.471132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.473778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 229/446] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.474987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.476705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.497756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c22018-7d0f-467e-bf78-40ff5687eec5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.538460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.539389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.539669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.539943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] PATH_INFO: `/v3/groups/91d670111e13408ab32d325a8f4e636c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.541533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] RBAC: Authorizing `identity:get_group(group_id=91d670111e13408ab32d325a8f4e636c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.543643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.545451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 218/447] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/groups/91d670111e13408ab32d325a8f4e636c => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.546287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.601557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f3a1-bdeb-4444-939a-e69b11cc1c32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.612396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.645525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.646314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.648491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.648491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.660053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:10.662986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509aea2-6857-4214-8f05-a6ab34ab8635 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:10.669490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:10.670350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:10.671137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 230/448] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:10.672371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.701145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.701793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.702073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.702343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.703167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=1685647e66434b21ac19c01987e91b17) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:10.703511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.704931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.711566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 219/449] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=986616b48b464a8c82fe68f806417c4a => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.711566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.732338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:10.791688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d944e9-8726-4d8a-ab2c-0b9930cbdb4c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.826713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbf77-8efa-4bf3-9d53-f98bda316adb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:10.831094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.831791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.832073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.832346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.833192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] RBAC: Adding query filter params (group.id=91d670111e13408ab32d325a8f4e636c, role.id=29fc77f10237449d82dfef29036abe51, scope.project.id=1685647e66434b21ac19c01987e91b17) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:10.833506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.834991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.851694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 231/450] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/role_assignments?group.id=91d670111e13408ab32d325a8f4e636c&scope.project.id=1685647e66434b21ac19c01987e91b17&role.id=29fc77f10237449d82dfef29036abe51 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.865206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:10.865925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:10.866205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:10.866477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:10.867302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:10.867614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:10.869066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:10.884293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 220/451] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:10.885794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:10.923935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:11.000941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8b98-1685-4d80-90c7-0e59dd446723 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:11.036700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:11.037355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:11.037633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:11.037905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:11.107885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:11.109387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:11.139387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:11.209744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 232/452] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:11.215170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:11.295604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:11.317886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fed84d-b226-490b-b74d-b8a6fdce973f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:11.352170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:11.353020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:11.353295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:11.353559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] PATH_INFO: `/v3/projects/fb9a8fcaeb99475786d754ce49440ef8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:11.355073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] RBAC: Authorizing `identity:get_project(project_id=fb9a8fcaeb99475786d754ce49440ef8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:11.356571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:11.358616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 221/453] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:05:11 2022] GET /identity/v3/projects/fb9a8fcaeb99475786d754ce49440ef8 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:11.363776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:11.462390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897d93d-9b5b-4a59-b3ed-bff2834b9b64 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:11.491381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:11.496734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:11.497600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:11.497876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:11.498147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] PATH_INFO: `/v3/groups/2eed0f41cfcd4212ae0056233ac487e6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:11.499307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] RBAC: Authorizing `identity:get_group(group_id=2eed0f41cfcd4212ae0056233ac487e6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:11.500745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a674c-b577-46dc-bba4-47d6b8631321 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:11.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 233/454] 10.222.0.39 () {56 vars in 1180 bytes} [Thu May 26 03:05:11 2022] GET /identity/v3/groups/2eed0f41cfcd4212ae0056233ac487e6 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:11.508056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:11.605846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598a0b7-61f9-40a5-bc95-c90b73f1f7b7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:11.637770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:11.638407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:11.638692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:11.638967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:11.639822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] RBAC: Adding query filter params (group.id=2eed0f41cfcd4212ae0056233ac487e6, role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=fb9a8fcaeb99475786d754ce49440ef8) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:11.640135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:11.641834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38f2f-45dd-40be-8150-44ec12c324c9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:11.647488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 222/455] 10.222.0.39 () {56 vars in 1376 bytes} [Thu May 26 03:05:11 2022] GET /identity/v3/role_assignments?group.id=2eed0f41cfcd4212ae0056233ac487e6&scope.project.id=fb9a8fcaeb99475786d754ce49440ef8&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:14.337257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23daa7b-7e94-4629-8f6c-8895d758e222 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.337809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23daa7b-7e94-4629-8f6c-8895d758e222 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.337809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23daa7b-7e94-4629-8f6c-8895d758e222 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.340249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 234/456] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:14.345821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f316a25-38c2-464b-9287-3e3eb39d47ed None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.346254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f316a25-38c2-464b-9287-3e3eb39d47ed None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.346421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f316a25-38c2-464b-9287-3e3eb39d47ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.372251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7f316a25-38c2-464b-9287-3e3eb39d47ed None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:14.373442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7f316a25-38c2-464b-9287-3e3eb39d47ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:14.379261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:14.469145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 223/457] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:14.483884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaa19-e7b4-4c75-8016-38c9d4dc6e85 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.484180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaa19-e7b4-4c75-8016-38c9d4dc6e85 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.484441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eaa19-e7b4-4c75-8016-38c9d4dc6e85 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.485814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 235/458] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:14.491048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.491366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.491646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.501016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eaa19-e7b4-4c75-8016-38c9d4dc6e85 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:14.509387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:14.510852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:14.622968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 224/459] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:14.657931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:14.671390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eaa19-e7b4-4c75-8016-38c9d4dc6e85 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:14.746868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:14.747943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.749741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.750162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.767266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:14.770836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:14.789650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:14.799939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:14.801020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:14.802062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:14.802062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:14.802062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:14.802062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:14.802062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 236/460] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:14.808319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:14.824781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fec3a2-5656-4147-938f-8e0d1bd120df None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:14.878552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:14.881639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.882195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.882747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.903679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:14.906927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:14.911339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-667d36f5-53f7-4485-8238-0c99cb87ed7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:14.938562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:14.939516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:14.940372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:14.940372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:14.940372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:14.940372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:14.940372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 225/461] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:14.941651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ea24-8f70-4c2f-a72e-a347323bf5c9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:14.941939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ea24-8f70-4c2f-a72e-a347323bf5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:14.942620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322ea24-8f70-4c2f-a72e-a347323bf5c9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:14.946292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 226/462] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:14.947266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322ea24-8f70-4c2f-a72e-a347323bf5c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:14.989982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223b1f9-10c9-48ec-8017-0683505dc493 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.032265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.033518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.033978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.034647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.035952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:15.036678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.038668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5c658-518c-4920-921c-841bce6da130 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.054060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 237/463] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:15.055562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.055852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.056129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.059667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322ea24-8f70-4c2f-a72e-a347323bf5c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.078657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:15.080010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:15.083512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:15.096348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.097081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.097430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.097768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.098475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:15.098780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.100281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ff87-e47a-4844-b15a-7f497a994625 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.131398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 227/464] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:15.131398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763feda-9eaa-403c-a761-df298aa80ee2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.131398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763feda-9eaa-403c-a761-df298aa80ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.131398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763feda-9eaa-403c-a761-df298aa80ee2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.132493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 228/465] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:15.135438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763feda-9eaa-403c-a761-df298aa80ee2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.165499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 238/466] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:15.167296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.232896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:15.255988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763feda-9eaa-403c-a761-df298aa80ee2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.294959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.295652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.295931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.296199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.296354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed050e-ae99-452b-aafa-b75d10daf7f2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.296949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.298427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece674af-d773-40ee-af6b-632dad161c6c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.327274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 229/467] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:15.327274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.327274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.327274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.343907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.344720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.344827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:15.345132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.345455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.346243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:15.346637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.347991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.378757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 239/468] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:15.379656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.425294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:15.501162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 230/469] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:15.527155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.589395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:15.593846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b12bb-490a-4638-b61e-f06401ed7db0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.639699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.641010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.641401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.641775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.654559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.656124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:15.681823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:15.683638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:15.684485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:15.684485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:15.684485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:15.684485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:15.686144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 240/470] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:15.692190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.701837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3f212-b038-4353-bc4d-7a041c27dff1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.751999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.752847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.753126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.753399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.767905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.770119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:15.790585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:15.792188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:15.794588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:15.794588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:15.794588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:15.794588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:15.794588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 231/471] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:15.798514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:15.822169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:15.881204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd678c-7c69-435b-975a-59b83a76fee5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.910742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6613d5-8a9e-43f2-9415-0ebe61eecc97 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:15.964011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.964601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.964877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.965147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.965879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:15.966180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.967596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a8f39-a1be-429f-bc66-c96b79760066 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.976035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:15.976718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.977004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.977281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.978016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:15.978324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:15.979836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f84794-64b0-4820-8dc4-88dcd3d3aac9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:15.991472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 241/472] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:15.992826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b28c4-a28d-41e1-ade7-72b9020ca68b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:15.993109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b28c4-a28d-41e1-ade7-72b9020ca68b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:15.993380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b28c4-a28d-41e1-ade7-72b9020ca68b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:15.995325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 242/473] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:15.996595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 232/474] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:15.998254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b28c4-a28d-41e1-ade7-72b9020ca68b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:16.001258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.001667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.001945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.011943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:16.021310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:16.022642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:16.102052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 233/475] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:16.103523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:16.127403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:16.134431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b28c4-a28d-41e1-ade7-72b9020ca68b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:16.180850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:16.181944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.182974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.183392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.184736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:16.186717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:16.224263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 243/476] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:15 2022] POST /identity/v3/users => generated 249 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:05:16.225269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:16.236542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfa5f9-b52a-44a6-8b1b-930b8c7cddb4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:16.277804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:16.278716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.279140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.279576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.280544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:16.282229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:16.321132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:16.346542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 234/477] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:16 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:16.366867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797802f-d40e-4445-856e-9e9defdd2cca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:16.409149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:16.409847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.410127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.410401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.411230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:16.412704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:16.424896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 03:05:16.424896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 03:05:16.424896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [parameters: {'id': '5fcc3bf7b3fa46c69182015be4a64fe0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 03:05:16.424896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:05:16.442693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return ref.to_dict() May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:05:16.443652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 03:05:16.445048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 03:05:16.446850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:05:16.449568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:05:16.457181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'id': '5fcc3bf7b3fa46c69182015be4a64fe0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:16.459140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 03:05:16.461169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:16.463132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 244/478] 10.222.0.39 () {60 vars in 1131 bytes} [Thu May 26 03:05:16 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:16.463525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:16.566858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:16.608906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d62519-1ef0-4d8b-a433-123d0c15a9ca None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:16.648609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:16.649433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.649715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.649990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.661346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f1533f1c-b0fc-4694-968d-7d250e542a84 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:16.669494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:16.671638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:16.673642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 235/479] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:16.676504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:16.793765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a26a6-df28-4c5f-a083-ce017737fddc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:16.826471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:16.833953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:16.835335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:16.836029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:16.836309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:16.837190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:16.837494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:16.839641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544f229-bcc6-4658-b528-2189edd0118e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:16.849308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 245/480] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:18.016726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e8f0-4d35-4fca-99a4-7863e93abf65 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.017520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e8f0-4d35-4fca-99a4-7863e93abf65 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.017520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e8f0-4d35-4fca-99a4-7863e93abf65 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.018954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 236/481] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:18.024786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.025287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.025388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.046220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:18.047576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:18.088031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946de89-8f0f-4cbe-ae93-5e5dac7c37a6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.089221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946de89-8f0f-4cbe-ae93-5e5dac7c37a6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.089813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946de89-8f0f-4cbe-ae93-5e5dac7c37a6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.092963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 237/482] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:18.102862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.103472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.104843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.128018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:18.130081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:18.177866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 246/483] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:18.223413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 238/484] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:18.232380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:18.244920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:18.371767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:18.390481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ccc4-57bc-485d-a6e4-a15e830a849b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:18.435110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:18.436870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.437734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.438469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.468071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:18.478621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:18.480105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:18.480910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:18.482082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 239/485] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:18.490125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:18.526340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabec92f-7491-4073-b6d9-7dcd42973d9e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:18.578813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:18.610336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:18.611143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.611452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.611732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.658043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96f486-afe6-4dc8-9f88-145f8c025f6b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:18.675991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:18.687114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:18.688230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:18.690556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:18.691401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:18.701278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 247/486] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:18.708167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:18.709074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.710372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.710814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.712001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:18.712271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:18.712627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:18.716893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:18.735364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 240/487] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:18.738192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:18.881239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f2599-bee2-4d3b-b18a-f40af0935227 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:18.929090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:18.930108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:18.930637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:18.931054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:18.945007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:18.946933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:18.975116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:18.977007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:18.978955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:18.978955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:18.978955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:18.978955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:18.986046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 241/488] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:18.987451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.030536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.117896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802ae5f-4037-40f4-9679-30ea04de2a25 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.122449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1347cc3-54d7-48c9-8b9f-49cd2bcdfc38 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.168849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.170395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.171074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.171821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.174314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:19.175082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.178073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.201650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 242/489] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:19.203325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.242664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.243645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.243645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.243952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.245218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:19.245531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.246921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.267623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 248/490] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 556 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:19.269357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.341180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f6b7e-956c-4629-beb5-394053ee9cbf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.359853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.380727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.384420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.385539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.386512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.387181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.399617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.401774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-71b0588c-6fc0-4c88-bdd3-be9c0bd9423f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:19.423878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:19.424856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:19.425832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:19.434271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 243/491] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:19.434271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1470-6d92-46bc-a5c7-d965496e1065 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.434271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1470-6d92-46bc-a5c7-d965496e1065 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.434271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1470-6d92-46bc-a5c7-d965496e1065 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.435112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 244/492] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:19.435903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc1470-6d92-46bc-a5c7-d965496e1065 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.463606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70905d6c-e0be-4d41-8f4d-fd3dd6823180 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.507047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.508542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.509096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.509610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.523723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.525887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-cdb0a30c-0174-4cb0-8bb3-a656686cf606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:19.544630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:19.545654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:19.546504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:19.546504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:19.546504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:19.546504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:19.546504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 249/493] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:19.547139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220f69-0080-4382-9038-907bee32354b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.547447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220f69-0080-4382-9038-907bee32354b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.547725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f220f69-0080-4382-9038-907bee32354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.554314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc1470-6d92-46bc-a5c7-d965496e1065 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.564656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4f220f69-0080-4382-9038-907bee32354b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:19.565842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4f220f69-0080-4382-9038-907bee32354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:19.602340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.603016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.603326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.603600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.605520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:19.605864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.607345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdd372-4bd8-407d-9cab-4c6b1f01c2a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.644643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 250/494] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:19.646276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1c6e-bc7f-48c2-a72a-7cfe113a7db9 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.646744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1c6e-bc7f-48c2-a72a-7cfe113a7db9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.647199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1c6e-bc7f-48c2-a72a-7cfe113a7db9 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.653415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 251/495] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:19.654399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f1c6e-bc7f-48c2-a72a-7cfe113a7db9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.661666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 245/496] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:19.663877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.664396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.664870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.686196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.691836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:19.694003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:19.696856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.770511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 246/497] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:19.771381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.783684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f1c6e-bc7f-48c2-a72a-7cfe113a7db9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.821123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.821791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.822074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.822348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.823152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:19.823545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:19.824932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:19.837071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 252/498] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:19.837888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:19.903018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47617ffa-32fc-420d-9390-07c4547e17c9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.937454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.938097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.938375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.938703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:19.943110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.948300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:19.958540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3eadb-6144-4c53-b53d-4d49851afd72 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:19.991316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297261cf-c935-4760-88b1-979a13281cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:19.992102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297261cf-c935-4760-88b1-979a13281cea None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:19.992378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297261cf-c935-4760-88b1-979a13281cea None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:19.992645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297261cf-c935-4760-88b1-979a13281cea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.003107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297261cf-c935-4760-88b1-979a13281cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:20.008189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.009665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-297261cf-c935-4760-88b1-979a13281cea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:20.010667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.011636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:20.012410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 253/499] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.013103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297261cf-c935-4760-88b1-979a13281cea None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.111792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297261cf-c935-4760-88b1-979a13281cea None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.113282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 247/500] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:20.114121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.145567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.146350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.146627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.146898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.154383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.157393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7660f0d6-22e4-47de-a036-99e18c7c4058 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:20.165001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.166048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:20.167619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 254/501] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.168521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24ab6-9a42-498a-9100-2a3d6105fd0e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.168724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24ab6-9a42-498a-9100-2a3d6105fd0e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.168993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24ab6-9a42-498a-9100-2a3d6105fd0e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.170237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 255/502] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:20.170918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b24ab6-9a42-498a-9100-2a3d6105fd0e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.180623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.226095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2f3fe-537c-46a3-a328-36d1ac609277 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.270816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.271640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.271917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.272180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.279467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b24ab6-9a42-498a-9100-2a3d6105fd0e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.279889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.281330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:20.296159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.297210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 248/503] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.298194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.315563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.316225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.316502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.316772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.317507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:20.317810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.319185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.327133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 256/504] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:20.328105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.396651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.403278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.406673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58e3d6-8b1c-4ab9-8f21-211163c552cb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.429934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad28b2-a798-4219-88e8-666318a922dd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.442715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.443528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.443809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.444083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.454427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.455909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.464252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.464909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.465189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.465462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.466204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:20.466508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.467897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-cf8af4e8-f73c-4114-a513-2ca2fdb8b135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:20.472822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.473832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.474869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:20.474869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:20.474869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.474869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:20.474869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 249/505] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.475196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.475522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.475778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.482901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 257/506] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:20.482901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.492593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:20.493774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:20.570112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 250/507] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:20.578417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.588574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b47e19-b655-4ff1-a81d-88bc00f68660 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.616378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.623194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.623820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.624094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.624365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.625199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:20.625501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.626926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.659500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.663259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 258/508] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:20.664085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.679202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5f47-26b3-4ee5-a9b7-c167895f022f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.713122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.714009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.714293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.714573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.724511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.725976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:20.744593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.745800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 251/509] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.746810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.766609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce986-4319-4966-94dd-4d463d486694 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.800647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.801305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.801588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.801867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.802585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:20.802898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.804331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.816003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 259/510] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:20.816792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.845011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c5fb7-4f62-4f1a-94c1-dcd41248734b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.865131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.874392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:20.879695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.880489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.880771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.881046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.892093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.893959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:20.913501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.914603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.915542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:20.915542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:20.915542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.915542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:20.915542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 252/511] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.915832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:20.926171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd752-fab3-4064-ad90-290f74af4bb4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:20.960728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:20.961756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:20.962033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:20.962303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:20.973995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:20.975476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:20.993812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:20.995458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:20.996486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:20.996486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:20.996486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:20.996486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:20.996486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 260/512] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:20.996766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.032224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f794ee-280c-4e21-b02f-0b8a79da330d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.067764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.068410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.068684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.068961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.095415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.098249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db6141-69ed-41ee-bf08-0d0e9ef1fe2c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.123796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.133942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.134708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.135064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.135376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.136095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:21.136403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.137768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.140882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.142340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.149531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 261/513] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:21.150330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.245128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 253/514] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:21.249756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.257662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0fc54-b771-4c7b-83b2-d442ac96cd00 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.293284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.294069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.294350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.294618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.301195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.302632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:21.317142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:21.317985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:21.318848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 262/515] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:21.319351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.329062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.342423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.345488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c4f5a-5e0c-48da-8e0a-b20dd2d54f9f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.381689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.382348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.382624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.382899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.383632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:21.383932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.385387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.396502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 254/516] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:21.397280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.424739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdf77e-fafd-4318-b1f4-4944e48051e8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.457727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.458377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.458659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.458926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.459665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:21.459965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.461338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.472852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 263/517] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:21.473628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.500528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d66e84-80f1-4456-9965-e44843bb220d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.533320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.534104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.534391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.534666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.543431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.544893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.554706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:21.571750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:21.573743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 255/518] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:21.575908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.580203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cf65f-53e1-4f8d-bc3e-4ba84768e910 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.581929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.616191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.616972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.617245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.617510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.628239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.629701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:21.647069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:21.647939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:21.648772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:21.648772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:21.648772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:21.648772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:21.648772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 264/519] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:21.649479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.683695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b3d6c-ee3c-4095-ac94-91cbc3527776 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.717718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.718374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.718657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.718930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.719731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:21.720036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.721458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.738988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 256/520] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:21.739811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.747019 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937df8a5-e308-4cd4-98ff-e01350b9e6ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.777605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.782733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.783574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.783856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.784130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.790978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.792576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.801006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:21.808955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:21.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 265/521] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:21.811028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.843216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb006d4-0139-4ced-bf1f-291e78d0c97b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.880132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.880924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.881091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.881366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.882071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.883543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.911922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 257/522] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:21 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:21.912674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:21.924765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb5a8-a851-499c-98fd-efa4c5ff5305 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:21.967642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:21.968332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:21.968614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:21.968884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:21.969747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:21.970058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:21.971618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:21.992852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 266/523] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:21.995053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.032251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d413a-78b8-4709-b230-d534a564ab3f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.049704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.055892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.074892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.075610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.075903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.076168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.076939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:22.077250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.078670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.095156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 258/524] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:22.095156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.111321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce51a6a-fa77-4958-88c4-4f174d114edd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.155530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5604366-d036-470c-a991-b661902099c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.156180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604366-d036-470c-a991-b661902099c9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.156493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604366-d036-470c-a991-b661902099c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.156757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604366-d036-470c-a991-b661902099c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.157589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5604366-d036-470c-a991-b661902099c9 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=7b48b2079573456c93a286b7d799e950) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:22.157898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5604366-d036-470c-a991-b661902099c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.159354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5604366-d036-470c-a991-b661902099c9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.165580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 267/525] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:21 2022] GET /identity/v3/role_assignments?user.id=7b48b2079573456c93a286b7d799e950&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:22.166385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5604366-d036-470c-a991-b661902099c9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.252375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0915-aafb-40be-a3a0-8e8b93940a25 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.300247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.301814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.302247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.302923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.304232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:22.304695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.307816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.309138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5604366-d036-470c-a991-b661902099c9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.325649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.342017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 259/526] 10.222.0.39 () {56 vars in 1139 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:22.343447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.351118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.352044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.352430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.352811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.406862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.439266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.441133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.482617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936aa-b154-45b5-bfc2-1280533478d5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.526892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.527870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.528164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.528450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.537452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.539472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.549649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 268/527] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:22.550768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:22.558140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:22.559049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:22.560242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:22.563315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 260/528] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:22.564341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.573265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.677950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615d26-a2af-4b30-91e3-f5ba74ba8c6a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.684701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.689549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80149bf-d648-474b-8ab8-4904cebd023a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.719826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.720486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.720824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.721103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.726914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.727740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.728020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.728295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.739994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.741468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:22.758792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:22.759724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:22.760615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:22.760615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:22.760615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:22.760615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:22.760615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 261/529] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:22.761331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.795212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.796745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.812011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:22.861221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e8ec-bb37-4ec4-a0ff-ea6b6fb76b55 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:22.895348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:22.896008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:22.896288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:22.896562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:22.897654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:22.897960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:22.899692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:22.905016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 269/530] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:22.906205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.918986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 262/531] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:22.918986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:22.963679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.027849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978bc2-166a-422e-b2ab-7e410c5d385c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.043331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.053911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694b697-f743-4a14-a495-49fcb5cfbeb5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.067004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.067702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.067986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.068265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.069304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.069423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.070810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.086131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 270/532] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:23.087075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.099714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.100877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.101393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.101930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.114627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.116641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:23.137525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:23.138412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:23.139232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:23.139232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:23.139232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:23.139232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:23.141889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 263/533] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:23.143017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.198829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638106ee-b95f-4710-bf24-d0563003a322 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.210430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.237809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.238471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.238755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.239031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.239967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.240282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.241960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.254347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd83f87-9672-49a5-8d6b-bb2244b1ead4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.255675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 271/534] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:23.256638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.293848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.295002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.295514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.296001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.303717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.305495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:23.325742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:23.326661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:23.327529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:23.332634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 264/535] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:23.340297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.340487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.418594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b9248-b4a0-477b-857a-956659310cec None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.476833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.477521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.478106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.478391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.479203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.479543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.480975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.511148 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 272/536] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:23.516752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.544099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1edb-2119-43ca-86d0-ced90877483b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.576611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.590669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.591365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.591653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.591926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.596935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.597262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.598719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.629534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 265/537] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:23.635635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.658560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9c1ed-4949-4f38-8fc6-9613409d3b7b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.679690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:23.699615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.700417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.700701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.700974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.717999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.721591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:23.753496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:23.754409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:23.755911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 273/538] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:23.762793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:23.845043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab36421-8813-4a9d-867a-fd00539213f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.913201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.913865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.914142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.914412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.915258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=5524a8ebd91f4e5eb7311923401c7798) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.915568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.917573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.929176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 266/539] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/role_assignments?user.id=5524a8ebd91f4e5eb7311923401c7798&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:23.935335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d0985-4b2d-4d5a-beb2-b3a5affb2667 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:23.972747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:23.973495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:23.973777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:23.974154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:23.974984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:23.975405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:23.977010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496df6b3-dd6a-4166-8512-320362761170 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:23.996629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 274/540] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:24.003970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:24.076018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:24.155018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1e881-2d9e-47ea-8048-6244a5f2b5a3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:24.200683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:24.201697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.202176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.202641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.203720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=15a31aebf6b8454cb69d574dbb0c6409) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:24.204227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:24.205846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d170408-8451-4c31-b326-a042bb8271a2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:24.212672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 267/541] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:24 2022] GET /identity/v3/role_assignments?user.id=15a31aebf6b8454cb69d574dbb0c6409&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:24.334464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a67c7-e734-4a3c-97ee-19adb1cc3f9c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.334974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a67c7-e734-4a3c-97ee-19adb1cc3f9c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.334974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a67c7-e734-4a3c-97ee-19adb1cc3f9c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.336642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 275/542] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:24.342488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.343474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.343968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.363419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:24.364935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:24.486956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 268/543] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:24.509612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a67c7-e734-4a3c-97ee-19adb1cc3f9c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:24.532833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:24.631546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a67c7-e734-4a3c-97ee-19adb1cc3f9c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:24.669594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:24.670401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.670682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.670957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.681887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4d8fbf69-1839-4e4a-9af5-7cf66285de9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:24.689701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:24.690587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:24.691412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 276/544] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:24.700353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:24.771959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:24.835076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87065e8d-2173-47ce-9ae6-cf87b804150e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:24.844413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c2b6-0fc1-42f8-9774-db07f945402c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.844998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c2b6-0fc1-42f8-9774-db07f945402c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.845504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c2b6-0fc1-42f8-9774-db07f945402c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.847370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 277/545] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:24.852656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.853183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.853823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.871691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:24.873229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:24.892909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:24.893516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:24.893792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:24.894061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:24.894805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:24.895107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:24.896520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:24.908767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 269/546] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:24.915828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:24.967027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 278/547] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:24.990254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.003626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.104163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc148f1-5932-42e6-970b-39a67ffab5c5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.150366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd23449-7d82-4c97-b661-bdddd78c13a4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.154757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.172253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.173384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.173861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.174348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.194770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:25.207705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:25.208590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:25.209455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:25.212131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 279/548] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:25.217862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.219441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.220239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.220516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.220786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.232477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.234559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:25.253726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:25.255118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:25.256186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:25.256186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:25.256186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:25.256186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:25.256186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 270/549] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:25.261341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.343932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.408488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c116aa-d94a-4330-8fe7-ccce57ea20ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.456000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.457056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.457532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.457997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.458938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:25.459529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.461133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.472271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b7dc5-b2ff-421d-aad7-bd71a304f648 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.472771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 280/550] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:25.480754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.535932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.537084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.537574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.538055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.539111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:25.539669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.541939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.574985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 271/551] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:25.580708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.598111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.608939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74951e3c-7a2b-453f-a4fb-ea42a4206252 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.651023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.656597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.657403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.657688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.657963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.671432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.672886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:25.697923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:25.699013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:25.700013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:25.700013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:25.700013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:25.700013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:25.701865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 281/552] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:25.705323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.775186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41d6fc-56ce-4949-89ab-03f6cc33a344 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.815896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.816687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.816964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.817228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.840460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.845567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:25.878244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:25.879224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:25.880189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:25.883721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 272/553] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:25.887732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:25.933021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4756f-a64b-428c-bdf6-041bdd28ed55 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:25.979180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:25.980234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:25.980867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:25.981592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:25.982208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:25.983149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:25.983722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:25.985331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654450-b35b-492f-8543-e911004ea03d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:25.999340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 282/554] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:26.001041 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a269f-f38e-48ac-bffd-dc00826edc4f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.001534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a269f-f38e-48ac-bffd-dc00826edc4f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.002017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a269f-f38e-48ac-bffd-dc00826edc4f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.003754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 283/555] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:26.006574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a269f-f38e-48ac-bffd-dc00826edc4f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.015156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a72e-bc34-48bb-983f-fc45cce03189 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.058002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.058863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.059263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.059647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.060646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:26.061520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.063084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedf017-f7a6-4c83-b9af-1e3c8b2f611b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.065007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.091341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 273/556] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:26.091341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.091341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.091341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.113661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:26.115192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:26.158653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a269f-f38e-48ac-bffd-dc00826edc4f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.189623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 274/557] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:26.190836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.196726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.198077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.198357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.198633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.207264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.208993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:26.227531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:26.229625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:26.230537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 284/558] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:26.234569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.262424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.303479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb04c1a-4f2d-431d-821c-d8118c371040 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.340106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.341162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.341648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.342123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.352928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d518a85-94a6-41c3-ab67-dbc479318a52 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.354740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.393624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.394722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.395154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.395593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.408249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:26.417153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:26.418349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:26.419403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:26.421870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 285/559] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:26.423015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.429819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.431466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.496008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.548530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6504e0-9d98-4f75-9269-5dbc590edf2c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.549468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 275/560] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:26.550506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.583047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.583614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.583995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.584372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.585386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:26.585818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.588674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.615298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 286/561] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:26.615298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.644515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.675829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcdc9-8eda-41d2-9fcb-833a2a78cf33 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.713988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.715044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.715565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.716063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.716990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:26.717577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.719144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.729440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 276/562] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:26.730567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.749470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c7000-9272-4aef-888d-d8a2ee64baa4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.758136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.794110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.794898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.795176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.795483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.806116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.808613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:26.826059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:26.827099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:26.828018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:26.828018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:26.828018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.828018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:26.828018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 287/563] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:26.828325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.846111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39c0a-1d41-4687-b9d5-85a88fb2bdc9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.879024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.879843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.880126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.880399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.892113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.893573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.896518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fee27545-d94c-444f-b6a5-2ae8e2d5f4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:26.910895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:26.911938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:26.912821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:26.912821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:26.912821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.912821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:26.912821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 277/564] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:26.913328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315251-f597-40c2-aaf1-2c38eff6e599 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.913614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315251-f597-40c2-aaf1-2c38eff6e599 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.913809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66315251-f597-40c2-aaf1-2c38eff6e599 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.919276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 278/565] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:26.919276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ceb78-2c8d-4930-b9f6-2f1aa21ab367 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.919276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ceb78-2c8d-4930-b9f6-2f1aa21ab367 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.919276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ceb78-2c8d-4930-b9f6-2f1aa21ab367 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.919276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 279/566] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:26.919668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ceb78-2c8d-4930-b9f6-2f1aa21ab367 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:26.931958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480c6f4-e802-4eca-976b-8d202c089c76 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:26.967861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:26.968655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:26.968938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:26.969216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:26.979079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:26.980552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:26.997794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:26.998887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 288/567] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:26.999889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.000511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.030861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ceb78-2c8d-4930-b9f6-2f1aa21ab367 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.064310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.064963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.065243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.065514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.066223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.066527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.067920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf798d-3e14-45e3-b5ad-bab240bda0d5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.080048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 280/568] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.081248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.081537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.081819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.097504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:27.098692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:27.099623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dad742-8a52-4447-9150-4a78dc409aca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.124377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.136149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.136806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.137089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.137366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.138087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.138394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.139821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32debd85-5307-496b-879f-e2fe9ada6886 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.152369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 289/569] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.153637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.153929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.154206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.170550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:27.171789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:27.172036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 281/570] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:27.172859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.238135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 290/571] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:27.238980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.255834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.292625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca76da-b44a-4215-b78a-beb49d3d626d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.325936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.326592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.326869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.327140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.327880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.328182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.329567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.341457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 282/572] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.342282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.349527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e5ea9-e9ac-4e35-baf2-46093ec469d5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.363892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.389704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.390742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.391022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.391318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.397791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.399228 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:27.418193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:27.419421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 291/573] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:27.420413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.448839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb276f-fade-4fd4-af9f-22cbc0f70024 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.473711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.483013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.483826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.484107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.484383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.490871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.492353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:27.506757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:27.508216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 283/574] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:27.509120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.529729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191b10-0917-40af-b691-21cfda25c315 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.564379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.565159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.565440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.565709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.575523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:27.583039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:27.584180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 292/575] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:27.585198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.611979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27de52-3505-4469-a5c4-5da0b43221c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.619118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.647217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.648020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.648301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.648578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.658548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:27.666642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:27.667723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 284/576] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:27.668678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.688151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698b175-f264-4774-a76a-00a3aa46b6c4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.715990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.721540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.722331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.722618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.722899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.730809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.732281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:27.746650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:27.747751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 293/577] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:27.753370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.772154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7c37f-30a2-4fd6-959d-2e59dfbdc2e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.805979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.807358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.807358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.807843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.808536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.808844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.810281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.821895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 285/578] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.822787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.845773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.852326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e91d-52ff-4d1d-825f-2eaf7789048d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.886986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.887671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.887946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.888223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.888996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.889300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.890999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.903861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 294/579] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.904661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:27.920508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1f9b-9618-40e4-8d7b-43d3c4643928 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:27.927687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:27.954914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:27.955601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:27.955890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:27.956167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:27.956903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:27.957217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:27.958572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:27.966411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 286/580] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:27.967185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.002834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abc8ba-d0ad-4ae9-98b0-eaba19aee0f1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.036948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.037552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.037831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.038096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.038830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.039128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.040539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.048945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 295/581] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.049721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.065295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c2a71-86da-456c-9594-50b3d127e912 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.069558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.100235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.100896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.101177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.101445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.102261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.102574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.104208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.121381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 287/582] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.122156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.148231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.151338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793641f9-cea2-4501-80b7-b34a6b528218 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.185486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.186146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.186434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.186706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.187569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=e6eb06c62ee340a0b70fbb88e9f3d09b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.187881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.189312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.194726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 296/583] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/role_assignments?user.id=e6eb06c62ee340a0b70fbb88e9f3d09b&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.195530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.231944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5c0f7-faa2-45fa-af0e-d3a43c46b5c8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.272225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.273237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.273517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.273816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/7b48b2079573456c93a286b7d799e950/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.289918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=7b48b2079573456c93a286b7d799e950, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.291889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.295833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.306566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 288/584] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:28 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/7b48b2079573456c93a286b7d799e950/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:28.307560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.331999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a853a-69c7-4050-bd69-fdc73eccdcdf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.382092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.383943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.384518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.385576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.399188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.401767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.408727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:28.427116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:28.428125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:28.429022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:28.429022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:28.429022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.429022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:28.431665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 297/585] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:28.432913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.441965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e577cd-7a11-4077-be43-4b28d2eb62ff None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.491293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.492159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.492442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.492714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.508744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.509361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:28.529302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:28.530448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:28.531554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:28.531554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:28.531554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.531554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:28.531554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 289/586] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:28.531892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.564921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9db86-4854-47de-b0f4-9eae72fbb64d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.608829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.615655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910f47-04a7-475c-ab02-279720408568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.616876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910f47-04a7-475c-ab02-279720408568 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.617257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910f47-04a7-475c-ab02-279720408568 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.617628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910f47-04a7-475c-ab02-279720408568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.632424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96910f47-04a7-475c-ab02-279720408568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.634412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96910f47-04a7-475c-ab02-279720408568 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-96910f47-04a7-475c-ab02-279720408568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:28.658339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:28.659317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:28.660410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:28.660410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:28.660410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:28.660410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:28.663302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 298/587] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:28.664276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910f47-04a7-475c-ab02-279720408568 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.666861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a566e8-bf01-4319-891f-458b83d7aa50 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.701860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.714656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.715361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.715648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.715922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.716645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.716956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.718331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.734396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 290/588] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.735270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.806237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96910f47-04a7-475c-ab02-279720408568 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.853944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.854952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.855476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.855966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.856788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d380-8ccf-4949-bc62-6d2537d386e9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:28.857108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.857603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.859199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.872704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 299/589] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.873847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.900004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:28.901525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:28.902040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:28.902507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:28.903463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:28.903980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:28.905531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:28.923364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 291/590] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:28.924541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:28.937136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:28.981110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.005700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c6a75-0000-4c7e-b07e-5e4062f9564b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.040827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c45b8-9d4f-4a09-a2ea-629a1c5c5f3c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.044905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.045747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.046035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.046312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.054097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.055751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:29.072121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:29.073146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:29.074707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 300/591] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:29.075363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.084342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.085522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.086006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.086490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.095063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.096991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:29.113991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:29.114903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:29.115871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:29.118584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 292/592] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:29.120273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.193220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a511-b65c-4d64-9c18-3658b70cbff7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.195838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.229358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dfe27-13d2-4b72-8f11-f2e1183081f3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.232830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.233727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.234013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.234290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.240957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.242270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.243893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:29.260649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:29.261549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:29.262442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:29.264988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 301/593] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:29.266335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.273868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.274621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.274974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.275434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.276344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:29.276725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.278323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.296970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 293/594] 10.222.0.39 () {56 vars in 1139 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:29.298034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.381760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efd56-f7e3-4e3e-9180-f9e965cfba66 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.409683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6dec5-a41a-4737-b9e3-87387fb4e16b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.420951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.422236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.422719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.423179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.424279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:29.424785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.426407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.449877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 302/595] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:29.449877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.450555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.451286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.451572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.451845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.452629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:29.452939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.454375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.472504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 294/596] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:29.473621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.479770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.507207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.568469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afb13-1e6e-4775-bc18-a9732fa7e902 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.586730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ebe85-271b-441d-b9d0-c1627ef1a873 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.606394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.607610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.608101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.608583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.619670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.621717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.623213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.624133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.624413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.624683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.635910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.637366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:29.643096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:29.643978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:29.644858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:29.644858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:29.644858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:29.644858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:29.647133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 303/597] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:29.648144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:29.656581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:29.657569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:29.658469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:29.658469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:29.658469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:29.658469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:29.662295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 295/598] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:29.663044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.729395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.755772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09a51e-75ae-4707-8f4c-8a75135843e1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.771846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3c9a-43fe-40e8-8dfb-1eb09e7bcbaa None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:29.792829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.793885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.794272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.794652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.795583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:29.810735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=2e9e7e21bd294ab389a68c50b4c2131a, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.811435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:29.812258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:29.812565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.812843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:29.813274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:29.814145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:29.814595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:29.816386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:29.830966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 296/600] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:29.832322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:29.835404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 304/600] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:29 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:29.836753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.006963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c09dd8-367d-4d69-b5bf-42b73a7a427f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.059437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755898b3-b8ec-4545-91a0-52e97b94da13 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.059995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.084784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.085879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.086161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.086433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.099052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.100596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:30.121759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:30.122708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:30.123637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 305/601] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:30.125035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.125947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.126430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.126923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.127880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:30.128394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.129986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.132393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.140699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.149147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 297/602] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:30.155512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.249194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0767be6-9f94-49a0-9401-1e70974984bc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.286335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.287361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.287928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.288427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.289593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:30.290109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.291857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.305259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 306/603] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:30.312264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.331055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f3fd2-6f4b-48c6-88f8-b9ed6019327e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.395592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.411123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.413447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.413942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.415092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.431050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.433662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:30.465620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:30.466579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:30.467528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:30.474059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 298/604] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:30.481454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.510488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614ba9-c7f6-4842-a02a-6cf2611e29d7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.564625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.566454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.566950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.567580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.567942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/15a31aebf6b8454cb69d574dbb0c6409/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.592460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=15a31aebf6b8454cb69d574dbb0c6409, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.596355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d9810-e6f8-47ad-bdfb-63e218258ac3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.615795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 307/605] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:30 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/15a31aebf6b8454cb69d574dbb0c6409/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:30.616416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9512c-c4fb-4198-8504-396ac71c3739 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.618395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ab4-4c30-4639-8916-5239d772a075 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.619795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ab4-4c30-4639-8916-5239d772a075 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.619795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e715ab4-4c30-4639-8916-5239d772a075 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.621420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 308/606] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:30.627487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a220-1c4a-494b-a8e1-34b8c0aaf010 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.627792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a220-1c4a-494b-a8e1-34b8c0aaf010 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.628045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998a220-1c4a-494b-a8e1-34b8c0aaf010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.651333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b998a220-1c4a-494b-a8e1-34b8c0aaf010 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:30.653236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b998a220-1c4a-494b-a8e1-34b8c0aaf010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:30.675143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.676294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.676789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.677292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.678429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:30.678980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.680713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.699595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 299/607] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:30.705703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.763977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 309/608] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:30.765730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e68d-826f-47fc-bb11-ea94714115b1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.766017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e68d-826f-47fc-bb11-ea94714115b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.766289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045e68d-826f-47fc-bb11-ea94714115b1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.767875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 310/609] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:30.773499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.773783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.774053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.793002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:30.794224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:30.815203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.858461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce7e0-939e-47e4-add3-d911810f70a8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:30.879345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 311/610] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:30.880230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:30.902147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:30.918322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:30.919606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:30.919899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:30.920176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/5524a8ebd91f4e5eb7311923401c7798/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:30.940175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=5524a8ebd91f4e5eb7311923401c7798, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:30.943031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:30.967268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 300/611] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:30 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/5524a8ebd91f4e5eb7311923401c7798/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:30.967268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.041889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b710e03-dbb2-4833-9abc-e6a30b483ff0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.079351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3f4da-829c-4895-8ec5-57ae85eb0a78 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.121199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.122011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.122291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.122561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.135653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:31.152215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:31.153322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:31.154677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 301/612] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:31.159786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.160670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.161719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.171450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.171831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.193789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:31.210064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:31.211071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:31.211925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:31.216110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 312/613] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:31.219573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:31.220472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.254882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:31.312784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919575a-ebea-4259-99fd-0cb42daee0f4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.350207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.350790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.351067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.351377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.352087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc60ae-912b-444d-9381-45f29bdfaf03 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.352213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:31.352405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:31.354510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:31.363289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 302/614] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:31.380533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.426668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.427691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.428116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.429955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.431415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:31.432110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:31.435008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:31.461832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 313/615] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:31.466178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.565815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:31.593153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf907e-bc67-4776-8fd1-04da47da0b16 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.603858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:31.631838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f340e-b21a-4c4c-885b-3423fefa8510 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.670048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.670892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.671180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.671489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.684488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:31.685974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:31.699200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.700646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.701134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.701618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:31.706826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:31.707695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:31.708697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:31.708697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:31.708697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:31.708697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:31.708697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 314/616] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:31.715813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.733227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:31.734699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:31.769078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:31.770000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:31.770869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:31.770869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:31.770869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:31.770869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:31.779687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 303/617] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:31.785214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.850015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ed082-62bb-430d-9bd0-54db08779a14 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:31.880978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:31.891646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:31.892772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:31.893432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:31.894093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:31.895432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:31.895851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:31.897386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:31.920243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 315/618] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:31.922493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:31.972001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066c19c-fb86-4033-8475-80db19fbcc68 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.037783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.038834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.041654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.044039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.044990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:32.046527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.048131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.067428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 304/619] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:32.069312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.073669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:32.177486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49481-f99c-4723-83c6-278d74446777 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.191287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07e23-d92d-47bc-a274-35bf2663afed None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.233032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.234566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.235070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.235557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.243867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.246582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:32.263202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:32.264186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 03:05:32.265116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:32.272334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 305/620] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:32.273794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.292007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.292816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.293156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.293368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.316508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.317995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.346531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:32.352548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:32.358260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:32.359205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 316/621] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:32.377564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.450551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7163c-d8bc-4a46-8a02-e0abd7f9b691 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.462387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:32.519096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d75a-496c-4343-ba09-7b2f032d6513 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.522753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.523465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.523754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.524321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.524856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:32.525205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.527071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53040733-415a-4a58-8e08-df1102bf5a6a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.557059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 306/622] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:32.558460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51bfa-3bf7-4097-a820-4bc08dc4ebb2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.558749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51bfa-3bf7-4097-a820-4bc08dc4ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.559024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f51bfa-3bf7-4097-a820-4bc08dc4ebb2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.560527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 307/623] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:32.562482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f51bfa-3bf7-4097-a820-4bc08dc4ebb2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.569909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.570590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.570872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.571145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.572462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:32.572783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.574818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ef268-55dc-4ed6-b5b5-cc1b45673d6b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.599153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 317/624] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:32.600606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.601004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.601405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.627528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:32.629169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:32.692516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f51bfa-3bf7-4097-a820-4bc08dc4ebb2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.715759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:32.716967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 318/625] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:32.718230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.733007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.733791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.734160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.734495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.764794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:32.829501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.831991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.833617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d807b-a738-4f2d-844c-e8a3cf00fdb5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:32.889614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:32.890414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:32.890945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:32.891225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:32.905357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:32.907319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:32.930117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:32.931764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 319/626] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:32.933843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.953040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 308/627] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:32.953951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:32.976477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.028806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.043696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260086d-cab8-4ed7-9180-240c14d78d20 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.067458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8d8e-3e7f-48ed-bb8f-72c137f69ccb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.080792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.081590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.081872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.082147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.093093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:33.100730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.101822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 320/628] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.103000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.110444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.112178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.112500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.112840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.113564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:33.113870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.115359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e871508-d046-4fad-8fdd-418cf24601c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.128832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 309/629] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:33.130133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dd933-4371-4653-a0e3-cbf03d6e8097 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.130420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dd933-4371-4653-a0e3-cbf03d6e8097 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.130694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dd933-4371-4653-a0e3-cbf03d6e8097 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.132104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 310/630] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:33.132619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dd933-4371-4653-a0e3-cbf03d6e8097 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.213301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c98ec-66cb-43d7-a0fa-3f2ec2a6098b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.232949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.248107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.248900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.249178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.249451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.255918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dd933-4371-4653-a0e3-cbf03d6e8097 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.261337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.262829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.271543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:33.283450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.284365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.285241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:33.285241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:33.285241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:33.285241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:33.287865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 321/631] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.288652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.295375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.296936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.297229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.297504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.298252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:33.298613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.300065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22faad4-271e-442f-8733-e23b3b42f397 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.310484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 311/632] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:33.311553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.311840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.312114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.329456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:33.330629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:33.405439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 312/633] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:33.406276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.407853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820cfa5-7b60-4966-b39f-7f007570d3b9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.451812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.452601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.452882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.453161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.461050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.462494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.480514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:33.481841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.482729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:33.484065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 322/634] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.484567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.520715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8ee6-0cf3-4dc6-bb31-f06ecfb4d0f5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.554691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.559425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.560246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.560529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.560802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.561519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:33.561823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.563198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.574520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 313/635] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:33.575355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.602478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcb4ac-403c-41c6-8179-bf61bf4d305e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.639772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.640818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.641092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.641357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.652037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.653629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:33.673401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.674325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.675189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:33.675189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:33.675189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:33.675189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:33.675189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 323/636] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.675505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.682865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ea314-1534-44d5-bcb0-cbe9aa0c7aa8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.712575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.718248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.719040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.719593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.719593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.730380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:33.738172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.739090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:33.739969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 314/637] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.740740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.779551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96318a1-4eef-4178-b14f-ebc2871e62b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.792246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.813964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.814628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.814907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.815181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.815991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:33.816297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.817760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59307c5-b88c-4d4c-9ef8-acc414e47f87 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.829296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 324/638] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:33.830539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517e37-e10e-40aa-8c78-f65171c8e615 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.830832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517e37-e10e-40aa-8c78-f65171c8e615 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.831113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517e37-e10e-40aa-8c78-f65171c8e615 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.832645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 325/639] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:33.833351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517e37-e10e-40aa-8c78-f65171c8e615 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.842873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c36f97-6359-4a2e-a563-1fb071b25f45 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.876124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.876911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.877192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.877464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.884173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.885814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:33.900610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:33.901551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:33.902457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 315/640] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:33.902940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:33.939035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517e37-e10e-40aa-8c78-f65171c8e615 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:33.954365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:33.971969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:33.972631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:33.972916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:33.973190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:33.973907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:33.974220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:33.975616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:33.987182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 326/641] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:33.987965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.006839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dfd6d-61b5-4ee9-a699-8a569bd291c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.021996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.045639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.046306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.046597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.046872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.047641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.047946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.049311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548b68b-e91f-495f-ba4e-e8fa654da9ba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.057668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 316/642] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.058835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.059122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.059428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.075378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:34.076559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:34.095223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f188f4a-4553-4bcc-b53b-46417c906583 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.128661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.129528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.129808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.130080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/e6eb06c62ee340a0b70fbb88e9f3d09b/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.143773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 317/643] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:34.144483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.156477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=e6eb06c62ee340a0b70fbb88e9f3d09b, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.157944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.171877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 327/644] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:33 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/e6eb06c62ee340a0b70fbb88e9f3d09b/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:34.172955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.192263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.256996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae84b-4688-49cf-99c2-213e2d49e818 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.286690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.296642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.297449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.297742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.298057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.298890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.299201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.300723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.320774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 318/645] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.320774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.321036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61699d81-1c5f-42e4-be00-29158af88751 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.359053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.359897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.360171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.360436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.373746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.375254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:34.390695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:34.391860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:34.393127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 328/646] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:34.393820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.443999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13586c9-76eb-4452-a50c-9c87a343a6e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.446302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.480691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.481479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.481760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.482028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.492544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.494131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.508778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b50e-7b39-4797-92f5-bb4c8538c7b9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:34.512067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:34.512943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:34.513845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:34.513845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:34.513845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:34.513845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:34.513845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 319/647] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:34.514120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.554024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.554846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.555126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.555786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.566620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:34.574745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:34.575733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 329/648] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:34.576609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.580600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.629882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8d03-dcda-47a0-9f40-4592f1d80fba None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.666433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.667306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.667590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.667859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.668679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=7b48b2079573456c93a286b7d799e950) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.668984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.670413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.676510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 320/649] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/role_assignments?user.id=7b48b2079573456c93a286b7d799e950&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.677529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.701256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf770b-a138-4fc1-a31c-3ee8f76a7eb5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.707655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.745990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.747117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.747801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.748418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.749553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.750094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.752355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.773965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 330/650] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.775774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.805079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc27e7-622a-4c39-bd58-866c585ac301 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.844254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:34.846419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.847408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.847908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.848327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.849284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.849794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.851527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.873184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 321/651] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.873184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.893317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c8cfb-328c-4810-af08-01f182099157 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:34.942679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:34.943389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:34.943933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:34.943933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:34.944670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:34.944955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:34.946329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:34.957266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 331/652] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:34.958868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:34.974990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.002028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9b3de-95e0-4e1f-9098-8eb02a78b099 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.051605 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.055256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.055768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.056275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.089594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a349306-d3c4-4145-85ed-5a961e0a4abb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.116058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.129784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.130998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.131619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.132243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.135292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.137269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.141988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.143755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:35.160837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:35.161789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 03:05:35.162788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:35.165719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 332/653] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:35.166883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.225528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.251222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 322/654] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:35.251222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.287478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fabf2-26ab-4600-abd1-0c6756466951 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.328363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.329665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.330142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.330706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.344170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.345967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.353551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c41cd-1e3a-42e7-a873-a9c93d0c4506 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:35.364983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:35.365874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:35.366747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:35.366747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:35.366747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:35.366747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:35.369642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 333/655] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:35.370716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.400631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.401592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.401858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.402130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.402963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:35.403285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.404981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.411635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.422737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 323/656] 10.222.0.39 () {56 vars in 1139 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:35.427870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.473182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.490316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc97a5a-1beb-482e-9ccd-7348dcaa5dd4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.528977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.530181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.530689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.531157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.535051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4e64b-0bfa-4dd0-b124-d3dd83896147 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.543705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.545517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:35.564971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:35.565870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:35.566742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:35.566742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:35.566742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:35.566742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:35.573507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 334/657] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:35.574722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.575139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.575947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.576321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.576688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.577506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:35.577907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.579461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.592581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 324/658] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:35.593661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.675781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.694274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3090a2-4bef-403f-8dc6-63bf8c3f446e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.699278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.704108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c677f8-caff-490a-b6ca-1cb7a099d1f4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.735251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.736650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.737091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.737513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.741518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.742554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.742554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.742978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.744382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:35.745032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.746558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.762944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 325/659] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:35.762944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.820670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.823005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.866650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b4ba-4ebc-40f1-9ff7-7f160bb5c92c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:35.903424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:35.904359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:35.904738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:35.905107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:35.912947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:35.914502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:35.919344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:35.933260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:35.934412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:05:35.935555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:35.937514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 326/660] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:35.938508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.942604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 335/661] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:35.943630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:35.965408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.053908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632936d-fed7-434e-b68d-d89d93c18f02 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.067781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c356d3-f041-4e73-88e9-9ba7891a0a32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.101144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.102601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.103457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.103960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.112060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.112842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.113268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.113787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.114753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.114963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:36.115399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.116731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.117074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:36.132950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:36.134002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:36.135283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:36.142207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 327/662] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:36.142207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.147232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 336/663] 10.222.0.39 () {56 vars in 1137 bytes} [Thu May 26 03:05:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:36.148246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.200387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.224129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.247443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2335a-0b19-4152-8b88-21682bac47c1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.264656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63760335-7c28-4b5b-97c9-d4799d16985e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.287481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.288448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.288828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.289347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.305682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.307855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.310027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.310951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.311421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.311847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.312771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:36.313217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.314781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:36.332415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:36.333343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:36.334219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:36.334219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:36.334219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:36.334219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:36.334482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 337/665] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:36.334864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 328/665] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:36.335703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.335918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.460743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edc3e1-3fc7-400c-bd98-abb8ca50cc68 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.471575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9599-db54-4f3c-87cc-6f6236a49622 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.475085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.491588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.497981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.498652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.498934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.499215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.500084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:36.500399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.501835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.515275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 329/666] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:36.515275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.523163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.523779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.524133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.524480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.608602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.611051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dabf3f-5db2-47f0-8694-94ee67325907 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.630485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ced80-7745-4b98-b84f-d8c8f0a7cb55 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.682126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.682960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.683370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.683741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.684682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:36.685087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.686673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2661cfc-c683-4b52-8268-a99f7791b68a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.700718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 330/667] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:36.702277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e770-b8f4-41d7-8de1-e51e19d2c67f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.702708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e770-b8f4-41d7-8de1-e51e19d2c67f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.703125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05e770-b8f4-41d7-8de1-e51e19d2c67f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.704875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 331/668] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:36.710410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e770-b8f4-41d7-8de1-e51e19d2c67f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.754857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 338/669] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:36.757622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.758155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.758635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.771006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.783033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:36.784287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:36.808599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:36.873418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05e770-b8f4-41d7-8de1-e51e19d2c67f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:36.876366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 339/670] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:36.878387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:36.917266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:36.918244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:36.918933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:36.920027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:36.929587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:36.932565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:36.954313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:36.955308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:36.956455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 332/671] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:36.957055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.002297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd870a1-c64d-4bba-a0dc-e67b66da7b2f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.041945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.042988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.043407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.043817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.046751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.059948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.062143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:37.082752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:37.083949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:37.084849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:37.084849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:37.084849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.084849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:37.084849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 340/672] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:37.085205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.103132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cd4ad-939b-4eba-8c20-7d5f64c7e975 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.117171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.141281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.142073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.142350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.142616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.154183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7e1053fd-9290-4ea6-8ce5-9c406e1ced76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:37.162431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:37.163398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:37.164232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 333/673] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:37.164935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64e36-1e68-4322-ad38-2249c1bc943e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.165566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64e36-1e68-4322-ad38-2249c1bc943e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.165566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c64e36-1e68-4322-ad38-2249c1bc943e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.167000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 334/674] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:37.170505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c64e36-1e68-4322-ad38-2249c1bc943e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.206066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c18735-3ef3-4ea1-88c5-e95240345262 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.243877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.244544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.244828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.245100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.246113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:37.246930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.248417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.266364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 341/675] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:37.266364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.289738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c64e36-1e68-4322-ad38-2249c1bc943e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.325201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.332108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.332847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.333130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.333403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.334125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:37.334428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.335849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2eeefe-37d1-41ff-b36e-c26eb90ec4d4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.347932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 335/676] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:37.353866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.354170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.354446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.374011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:37.375227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:37.391424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40522eae-d17c-448e-98c4-800cdddc66f7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.403106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.427864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242b60-174f-439d-a700-0568c6495083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.428462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242b60-174f-439d-a700-0568c6495083 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.428746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242b60-174f-439d-a700-0568c6495083 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.429022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242b60-174f-439d-a700-0568c6495083 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.429771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242b60-174f-439d-a700-0568c6495083 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:37.430082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242b60-174f-439d-a700-0568c6495083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.431510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242b60-174f-439d-a700-0568c6495083 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.444074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 342/677] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:37.444074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242b60-174f-439d-a700-0568c6495083 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.451026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 336/678] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:37.459257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.550483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242b60-174f-439d-a700-0568c6495083 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.560028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.572910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726588d-93c9-451e-b5ef-6958f776aad6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.586294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.586955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.587230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.588109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.592220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=15a31aebf6b8454cb69d574dbb0c6409) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:37.592220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.592220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.598833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 343/679] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/role_assignments?user.id=15a31aebf6b8454cb69d574dbb0c6409&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:37.599513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.634085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.636351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.637098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.637578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.648295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.650714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:37.674030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:37.675213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:37.676167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:37.682715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 337/680] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:37.682715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.719180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.728145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6284569-7ee9-4e71-8458-48c577f2eb1f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.778278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.779096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.779427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.779701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.792525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.794102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:37.813959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:37.815068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:37.816260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:37.816260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:37.816260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.816260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:37.816480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 344/681] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:37.818213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.818835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214a19d-5412-4734-b23e-193317cac457 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.858478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:37.867215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:37.868086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:37.868370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:37.868690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:37.881787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:37.883274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:37.905236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:37.906170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:37.907127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:37.907127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:37.907127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:37.907127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:37.907127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 338/682] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:37.907972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:37.968025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63cda3-61d1-4bb0-914a-b3ff5234a680 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:37.995649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:38.003261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.003952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.004238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.004515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.005310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:38.005623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.007070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.021127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 345/683] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:38.021873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d258a2-b076-4aea-81f0-af99ea05b96b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.021984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.071428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.073616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.073616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.073616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.074300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:38.074609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.076027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.089498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 339/684] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:38.090462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.126898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:38.144092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0941-9864-4fc8-be30-091b10bdd67e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.180019 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.180570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.180865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.181290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.182050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:38.182346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.183857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.200695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 346/685] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:38.201671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.202415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72fc4-b321-49e1-bf9c-76a894e0d809 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.245870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.246966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.247481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.247956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.249405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=5524a8ebd91f4e5eb7311923401c7798) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:38.249926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.252250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.256712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:38.267797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 340/686] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:38 2022] GET /identity/v3/role_assignments?user.id=5524a8ebd91f4e5eb7311923401c7798&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:38.269575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.373742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282d2d6-6d7c-4548-a8bd-d30c1c1a3c05 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.427460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e9f72-4de9-4137-b99c-78435a35668c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.442393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.443605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.444093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.444569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.459086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.459853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.460134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.460414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.460769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.461123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.463091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.463450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:38.479774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:38.480818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:38.481747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:38.484685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 347/687] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:38.491536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.500858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:38.505137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 341/688] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:05:38 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:38.617362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884b6e-a785-42e3-b354-a1ad81d73246 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.656538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.657218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.657499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.657776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.658624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:38.658933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.660420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.680787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 348/689] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:38.686835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:38.755678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:38.799808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1783-4948-41b7-ab2d-4850cd6e4c89 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:38.836342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:38.837236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:38.837781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:38.838197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:38.919012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:38.921288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc31a-e845-48aa-8218-06849e060c47 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:38.984016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:39.033902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 342/690] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:39.039869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.105030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:39.110431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72e54-b496-445c-8b2a-47a6d902a3bf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.110727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72e54-b496-445c-8b2a-47a6d902a3bf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.110995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72e54-b496-445c-8b2a-47a6d902a3bf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.112671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 343/691] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:39.123318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.123602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.123879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.152068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:39.153271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:39.171044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda24e19-9948-4dcc-bde8-025575e319bb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:39.214414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:39.215221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.215533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.215808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.236210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:39.237678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:39.271712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:39.272646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:39.273548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:39.273548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:39.273548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:39.273548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:39.274547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 349/692] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:39.279118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 344/693] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:39.284276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.341790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.465608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8e6a2-de00-4e6b-84f4-c9c28f5728b7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:39.494294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:39.494723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:39.521399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289820-5a95-4df3-b518-46d1b00478ad None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:39.535883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:39.536480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.536763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.537037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.537782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:39.538090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:39.539494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:39.564450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 350/694] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:39.565332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.594268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:39.595073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.595378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.596567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.617506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:39.625205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:39.626157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:39.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 345/695] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:39.635518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.743852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962fd8e-6398-42d8-89cc-647105e1e1f6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:39.780492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934b0c9-08ab-41d9-bd93-fde0f2f55810 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:39.791514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:39.792124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.792406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.792682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.793436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:39.793743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:39.795142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:39.810411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 346/696] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:39.816232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.835641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:39.846665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:39.847517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:39.847804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:39.848080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:39.856178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:39.857808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:39.874566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:39.875603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:39.876485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 351/697] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:39.882100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:39.927597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.002419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68087ece-5132-4ebd-abea-ecfe6d14549c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.038519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.039195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.039507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.039783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.040645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:40.041365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.042569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.045714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26d76c-f24d-4fa6-b88b-636230a8a4d7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.055178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 352/698] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:40.063990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.091368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.092499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.092985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.093460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.109753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.111215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4e42d809-ff4f-4065-9427-9798d4ffb3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:40.138316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:40.139274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 347/699] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d9b5-bd6e-4327-81e6-3a69710c384d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d9b5-bd6e-4327-81e6-3a69710c384d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d9b5-bd6e-4327-81e6-3a69710c384d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.144044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 348/700] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:40.148041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73d9b5-bd6e-4327-81e6-3a69710c384d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.223486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.232036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0f47-fbed-4647-9f83-af2561a9e068 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.248125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.272881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.273828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.274403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.274972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.276136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=e6eb06c62ee340a0b70fbb88e9f3d09b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:40.276672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.278300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e4d7f-d6ea-444f-87d2-225cc7cdbe3e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.286707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 353/701] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/role_assignments?user.id=e6eb06c62ee340a0b70fbb88e9f3d09b&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:40.288468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.288958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.289530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.303977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73d9b5-bd6e-4327-81e6-3a69710c384d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.312775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:40.313977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:40.362794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.363455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.363736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.364005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.367234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:40.367234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.369324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.411407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 349/702] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:40.426474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.439367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 354/703] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:40.508687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.572528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c5c89-510b-4769-9bdb-f1084ba151f4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.598826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.619367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.620375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.621583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.621966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.622275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.631810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.634693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.637830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d72c4-1a57-4aca-b137-a802ed2315e2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:40.655022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:40.656311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:40.657451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:40.663261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 350/704] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:40.665301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.686754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.687416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.687984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] PATH_INFO: `/v3/services/network` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.688697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.690134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:40.705193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:40.706241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:40.707612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 355/705] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:40.713468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.800689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead73b3d-d058-40c3-a5b2-1abd6b2f58e2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.835499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee961e-0175-4aaa-8fc7-a3ad26eef8eb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:40.846497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.848009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.848922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.849547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.850735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:40.851266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.854666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.877844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:40.878515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:40.878798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:40.879075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:40.882508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:40.882508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:40.882508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:40.883066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 351/706] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:40.886346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.888740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 356/707] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:40.895828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:40.919248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:40.940156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.041783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122e599-e8cd-40a3-a209-471d2254e9c6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.052840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f077d-6c31-4e5a-b5e2-0602632b90c2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.122910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.123604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.123794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.124076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.130893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.132093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.135141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.135736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.136718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] RBAC: Adding query filter params (type=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:41.142958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.144779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f184a-3665-4523-a3f9-881b84f3ee43 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.154724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 357/708] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:41.247353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e64a-b8b8-4790-addc-138978aab459 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.247683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e64a-b8b8-4790-addc-138978aab459 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.247921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e64a-b8b8-4790-addc-138978aab459 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.249677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 358/709] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:41.255181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.255516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.255811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.256089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.256688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.278279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:41.279531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:41.339647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.404274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 359/710] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:41.421767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 352/711] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:41.428375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:41.429593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:41.440553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.535607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8356-a361-4d7c-a3b2-70c41aa21b45 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.585025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.586470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.586930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.587542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.599742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.601253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f92943fe-640b-4f60-be72-8d72927d69ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:41.622118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:41.623080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:41.623990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:41.623990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:41.623990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:41.623990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:41.627535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 360/712] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:41.629136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3afce-9ec6-403c-aae0-ce388744510c None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.629426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3afce-9ec6-403c-aae0-ce388744510c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.629704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3afce-9ec6-403c-aae0-ce388744510c None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.631443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 361/713] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:41.633207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3afce-9ec6-403c-aae0-ce388744510c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:41.703276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.716854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c0f7-3b78-4b83-8416-16a9494479c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.751856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3afce-9ec6-403c-aae0-ce388744510c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.771095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.771793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.772245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.773452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.775664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.791210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.792024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.792410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.792786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.793675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:41.794108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-dd742124-e3e8-460e-905c-b50b2327f77e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:41.795400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:41.796570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:41.797585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:41.798536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.805249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 353/714] 10.222.0.39 () {56 vars in 1129 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:41.805249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.805249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.805249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.812269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.818729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 362/715] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:41.820026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:41.828085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:41.829334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:41.913830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 354/716] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:41.915098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:41.947511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc12069-ea01-4e84-b25b-8bd9f9701a8c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:41.971702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:41.983809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:41.984447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:41.984729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:41.985005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:41.985785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:41.986094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:41.987553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:41.992515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 363/717] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:41.993303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.037872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62b267-1595-43be-b2cc-16137235f894 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.073259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:42.079420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.080885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.081584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.082149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.090308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.092246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:42.107475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:42.108471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:42.109390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:42.112043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 355/718] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:42.113139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.125859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3d0a-d8cb-4699-9846-8431236166b2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.170463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.171273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.171566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.171845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.186044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:42.195798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:42.196832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:42.197680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 364/719] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:42.198367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.227441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca260b-0ece-4fee-a17d-b18cfd1ac962 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.260937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:42.267025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.267954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.268472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.268946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.269868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:42.270375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.271965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.284524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 356/720] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:42.285893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.325685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8639e75-1dca-4739-839f-ae5e692bf030 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.378473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.379279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.379648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.379999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.380953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:42.381339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.383033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.401042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 365/721] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:42.407476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.420059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:42.525420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f14fd2-4712-41a0-99c3-94f920e47fac None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.589253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.590785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.591275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.591698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.592797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:42.593380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.595227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.606075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 357/722] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:42.612577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:42.626883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179654-aa1f-4e1c-8d5d-26b67faa638c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.639717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:42.692971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.693626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.693902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.694173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.694998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:42.695327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.697767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.708769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 366/723] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:42.770323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ca29-df63-49d9-9470-61abbd6d1c5f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:42.792545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:42.816533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:42.817337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:42.817615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:42.817884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:42.840286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:42.841750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:42.862828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:42.864004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:42.865027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:42.865027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:42.865027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:42.865027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:42.865027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 358/724] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:42.870399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.036537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879945-b72d-44e0-9927-562f6a8c8165 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.077868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.078452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.078732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.079012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.079948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:43.080073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:43.081450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b429-ec42-4371-a3d5-766fe3de7e0f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:43.094260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 367/725] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:43.100485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.255786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830030-f05e-4856-807c-554f7ea57a6a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.256340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830030-f05e-4856-807c-554f7ea57a6a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.256597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec830030-f05e-4856-807c-554f7ea57a6a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.259911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 368/726] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:43.263904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b8ee8-cc04-4d56-b318-baa291b14386 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.264346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.264504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.264876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.288173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:43.289367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:43.331000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.331751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.332032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.332304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.339635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:43.343577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:43.345035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:43.376433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:43.377375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:43.378299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 359/727] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:43.383064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.417503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:43.477102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 369/728] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 03:05:43.496265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715a046-3d21-4d9c-b167-f855a5ca6f23 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.518076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.532920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.533599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.533880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.534156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.534999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:43.535334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:43.536769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:43.555638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 360/729] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:43.563093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.695936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ca373-da3b-4a55-a6bd-0b24ded18f4f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.714858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:43.730267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa784d-3c58-47db-aad8-40c0ab030ee5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.735424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.736431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.736891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.737340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.749328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-26ee3581-c409-46bc-9b98-b1d8cb40fecd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:43.757892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:43.758893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:43.759855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:43.763001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 370/730] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:43.764836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d9b0-0413-4873-85c4-e7c7049e6994 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.765599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d9b0-0413-4873-85c4-e7c7049e6994 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.766076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482d9b0-0413-4873-85c4-e7c7049e6994 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.768019 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 371/731] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:43.768959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482d9b0-0413-4873-85c4-e7c7049e6994 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.780367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.781038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.781322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.781598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.794680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:43.863588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:43.865284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:43.903320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482d9b0-0413-4873-85c4-e7c7049e6994 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:43.949120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:43.949714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.950254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.950539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.951320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:43.951633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:43.953711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d06f51-cb65-410c-88cb-c46cfc4d6a27 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:43.960000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:43.967081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 372/732] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:43.968621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:43.968911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:43.969184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:43.982199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 361/733] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:43.983410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:43.992712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:43.994048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:44.075986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 373/734] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:44.077347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.106105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:44.122709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f201d37-d964-4ed1-8bef-d429ca9ee138 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.171474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.172276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.172559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.172833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.186949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.188948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.203692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064897d-4fbf-4987-8704-fee5ca2c49d7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:44.215166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:44.216093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:44.216977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:44.216977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:44.216977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:44.216977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:44.217693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 362/735] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:44.223260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.240598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.241405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.241686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.241959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.252807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.254334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.271862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:44.273973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:44.277166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 374/736] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:44.279335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.345231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:44.356383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5752-9519-44e6-a4d9-1336c0146384 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.397032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a436f-7089-4027-a1ba-8c1fee0b726c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.401504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.402366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.402797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.403265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.404135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.405669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.435617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 363/737] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:05:44 2022] POST /identity/v3/services => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:44.436851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.450880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.451632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.451914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.452233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.452980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:44.453295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.455414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.471585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 375/738] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:44.476796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.603321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e0b81-93cf-4ce8-8b40-532fde31082b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.629203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:44.647439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.648481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.648766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.649040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.658950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.660611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:44.676605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:44.677624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:44.678573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:44.682925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 376/739] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:44.685554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.723985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:44.729785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39462a-3d40-4d24-b26a-80aa64d1c8d5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.774058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.774816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.775103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.775473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.776243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:44.776556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.778071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.795218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 364/740] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:44.799839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.820824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db95375-90b2-4e9b-97c5-116706f9a077 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.870548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:44.871227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:44.871535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:44.871808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:44.872655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:44.872963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:44.875566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:44.902569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 377/741] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:44.904139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:44.959985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851e7d2-d3a1-4388-9c87-aa4a80dd55d7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:44.979775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.014273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.015720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.016231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.016980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.029546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.031530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2795bc5f-a73c-4288-aabf-6b17fdd568ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:45.048824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:45.049770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:45.050657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:45.051081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1cb59-bcdd-4315-a1da-21a76f0dde6e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.054052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 365/742] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:45.055572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018663-ebea-4f50-9403-0c8821d96125 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.056691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018663-ebea-4f50-9403-0c8821d96125 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.057162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c018663-ebea-4f50-9403-0c8821d96125 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.058974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 366/743] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:45.061554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018663-ebea-4f50-9403-0c8821d96125 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.094848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.098734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.099437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.099721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.099995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.185839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.187557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.205746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c018663-ebea-4f50-9403-0c8821d96125 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.231272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.251779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.253154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.253902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.254332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.255623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:45.256495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.258632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b37c2-82e3-43f8-893e-cede5c071a94 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.276818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 367/744] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:45.279482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a4516-19ed-402a-87f0-d35566211576 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.279922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a4516-19ed-402a-87f0-d35566211576 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.280345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a4516-19ed-402a-87f0-d35566211576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.302474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-707a4516-19ed-402a-87f0-d35566211576 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:45.303931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-707a4516-19ed-402a-87f0-d35566211576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:45.304310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 378/745] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:44 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 03:05:45.305638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.395162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 368/746] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:45 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:45.396572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8cdf6-f7b8-4f4f-a65e-2cb2c450fff4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.396935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8cdf6-f7b8-4f4f-a65e-2cb2c450fff4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.397248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8cdf6-f7b8-4f4f-a65e-2cb2c450fff4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.398649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 369/747] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:45.399278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8cdf6-f7b8-4f4f-a65e-2cb2c450fff4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.413528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.433089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c450-8dea-40db-9d8a-387d1aaf5975 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.471374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.472075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.472381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.472675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.473445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=41c0ab6341464ac388e96852bb400679) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:45.473756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.475278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0ccf1-478b-4e3a-a492-1c8874f9808f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.488784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 379/748] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/role_assignments?user.id=41c0ab6341464ac388e96852bb400679&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:45.488784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.488784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.488784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.492290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.504280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:45.505760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:45.528056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8cdf6-f7b8-4f4f-a65e-2cb2c450fff4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.584464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.585275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.585553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.585821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.597251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 380/749] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:45 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 03:05:45.599009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.607292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.607292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:45.631510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:45.632634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:45.633609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:45.633609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:45.633609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:45.633609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:45.633609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 370/750] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:45.634411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.709897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.738663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd739313-0342-414f-8d18-4fd76b6ea2ec None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.754134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5801ac-b8d7-42f4-ae18-fea0f9038187 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:45.779402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:45.786681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.787945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.788328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.788699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.790789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:45.792599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:45.794998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:45.796017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:45.796485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:45.797016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:45.809349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:45.819422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 381/751] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:05:45 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:45.819422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:45.829414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:45.830444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:45.831326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:45.833706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 371/752] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:45.842113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.018688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bb85d-8e9d-4630-aa78-9f4433dc7369 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.035351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c674-9e21-488c-a558-c22060dacd11 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.060821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:46.074847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.075835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.076266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.076888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.077789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:46.078141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.078445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.079002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.079407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.079778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.080496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.080700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:46.081097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.082740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.090940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 372/753] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:46.099275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 382/754] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:46.100715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.101702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.171017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:46.242983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc00b3-8be7-4c97-bbfb-371251aa7b69 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.260493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e65c1a-3c7b-4f01-996a-b9a5b1d8161a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.305170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.306446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.306818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.307096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.326020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.326823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.327106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.327409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.331199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.332817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.340761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.343206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:46.350732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:46.351655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:46.353057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:46.353057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:46.353057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:46.353057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:46.353057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 373/755] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:46.360183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:46.365401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:46.366260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:46.367254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:46.370083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 383/756] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:46.379305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.448528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:46.541318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3705e-5a88-492c-96e5-a9009ab8c740 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.578235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b750d-6675-4c86-87ee-d64e69760c55 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.597622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:46.608365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.609042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.609326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.609603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.610336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:46.610644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.612044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.635599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 374/757] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:46.639075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.659868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.660543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.660819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.661095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.661894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:46.662204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.663684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.688132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 384/758] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:46.691679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.793603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11715a7-dee8-42a5-9a4b-c83bdbedfb4a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.804735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:46.862100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.862820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.863102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.863400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.881822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.883296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:46.907214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:46.908195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:46.909132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 375/759] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:46.921023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:46.932804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f4ac6-4349-4703-96ab-62f6b1f6e40a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:46.974804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:46.977276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:46.978252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:46.978529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:46.979434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=e6eb06c62ee340a0b70fbb88e9f3d09b) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:46.979750 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:46.982315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:46.997089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 385/760] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/role_assignments?user.id=e6eb06c62ee340a0b70fbb88e9f3d09b&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:47.076285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d9486-d111-4d80-bc5a-761e3d64d19e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.117774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.119075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.119476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.119756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.120612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:47.120924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.122388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba97d0b-a35c-40fa-bd76-0a855fc71e82 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.147054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 376/761] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:47.153088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.200383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:47.249870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055fbc2-bf51-4097-9dfc-ea9f2f8cbac8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.250188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055fbc2-bf51-4097-9dfc-ea9f2f8cbac8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.250424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055fbc2-bf51-4097-9dfc-ea9f2f8cbac8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.252081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 377/762] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:47.257562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.257843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.258123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.288605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af64c4-6547-41d2-b41f-2a9dece79813 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.289823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:47.291716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:47.313761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:47.330996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.332476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.332953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.333410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.346489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.348419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:47.382564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:47.383610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:47.384691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:47.384691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:47.384691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:47.384691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:47.384691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 386/763] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:47.392212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.410618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 378/764] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:47.434777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.540753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43444530-33a4-4f5e-8bbf-e0f4afbb2d35 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.579307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.580117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.580401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.580675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] PATH_INFO: `/v3/services/network` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.581333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.582704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.601316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:47.604925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:47.605972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:47.606829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:47.607650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 379/765] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:47.615090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.627697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bf5b3-dd06-4a79-b2ca-2989efc2b476 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.637623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:47.669685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.670765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.671286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.671766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.672993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:47.673509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.675181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.689528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 387/766] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:47.695165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.849399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511579-c98a-4020-9069-7ce616743849 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.869111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3fd6-8bc6-4bc6-a55a-26e189d05980 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:47.911271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.913108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.914723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.915999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.932050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.934205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.936203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:47.937030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:47.937452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:47.938029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:47.939948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] RBAC: Adding query filter params (name=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:47.940724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:47.942946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:47.958545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 380/767] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:47.959365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:47.960406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:47.961509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:47.964055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.968942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 388/768] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:47.970675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:47.974940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.082036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfed6b-0971-4a4d-9a81-fbd6f9f7c7f2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.120095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.121139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.121629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.122097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.123083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:48.123607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.124298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.126438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.145287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 389/769] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:48.146995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.191546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d10a-03fe-4574-b068-72bcc634880d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.213575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.246514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.247584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.248067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.248534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.249425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] RBAC: Adding query filter params (type=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:48.249960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.251545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd170f-985f-4cc9-b057-fdba75dad6ee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.257211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 381/770] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:48.257874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e6eb-cd09-4b2e-af2e-b66ff1cc9033 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.259097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1181f-c28f-4f56-b583-172419396b81 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.259853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1181f-c28f-4f56-b583-172419396b81 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.260354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1181f-c28f-4f56-b583-172419396b81 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.262445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 382/771] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:48.263400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1181f-c28f-4f56-b583-172419396b81 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.312250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.313166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.313447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.313730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.316780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=2e9e7e21bd294ab389a68c50b4c2131a, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.319359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0560d5-ecab-4e05-9e31-5018b0e73fae None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.341863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 390/772] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:48 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:05:48.343570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208ed9-1d3f-4a45-bc65-3b86cf8e2cf6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.343570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208ed9-1d3f-4a45-bc65-3b86cf8e2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.343833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208ed9-1d3f-4a45-bc65-3b86cf8e2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.398804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-49208ed9-1d3f-4a45-bc65-3b86cf8e2cf6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:48.401002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49208ed9-1d3f-4a45-bc65-3b86cf8e2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:48.459887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1181f-c28f-4f56-b583-172419396b81 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.499552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.500574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.501133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.501598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.502599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] RBAC: Adding query filter params (interface=public, service_id=1e91b859a7854cf08211203d19f2783d, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:48.503109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.504717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44bf02-7ac6-458b-89b5-8f83bc614ad5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.513984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.516948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 383/773] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/endpoints?service_id=1e91b859a7854cf08211203d19f2783d&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:48.519591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93d98-fb37-4415-b661-62b587b56c86 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.520254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93d98-fb37-4415-b661-62b587b56c86 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.520749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d93d98-fb37-4415-b661-62b587b56c86 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.522890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 384/774] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:48.524135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d93d98-fb37-4415-b661-62b587b56c86 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.537147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 391/775] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:48.538536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.538829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.539108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.557299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:48.558541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:48.616277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.644034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d93d98-fb37-4415-b661-62b587b56c86 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.645281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 392/776] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:48.647348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.699033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.699832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.700116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.700442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.701130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.702606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.708742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 385/777] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:48.709626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.788113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:48.797114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ac408-44d2-432d-97d6-9f4a30ca1a19 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.837621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.838797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.839169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.839389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.844073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d5c3-ee0b-4726-b4d3-56cca96e369e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:48.850584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:48.859023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:48.860214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:48.861068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 393/778] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:48.866399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.894821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:48.896014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:48.898191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:48.898709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:48.900440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:48.903905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:48.931559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:48.932739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:48.933666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:48.943397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 386/779] 10.222.0.39 () {56 vars in 1129 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:48.949838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:48.970399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.053028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2403a8-46ec-4f9a-a83d-7096a04bf9bf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.103507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.104198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.104332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc005d8-788d-41fd-b90e-79905242649e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.104509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.104750 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.105538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:49.105925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.107460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.122624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 394/780] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:49.124001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.133404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.145070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.145759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.146040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.146312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.147102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:49.147442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.148904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.156251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 387/781] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:49.164461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.339292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1d39-ba8c-422b-8a4e-da0b4a4dfaba None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.375482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.376434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.376870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.377294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.378151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:49.379197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.380735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.387151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 388/782] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:49.393326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.479215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf65621-6f85-482a-bfdb-3f462f84f3ad None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.490777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.495613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.527312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.528616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.529049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.529471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.543469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.545779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.552842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08b20c-89a5-4747-8ad7-0ed8714fdc5b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-b3f7668b-f869-4675-8ca2-c338937cf9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:49.570463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:49.571800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:49.572911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:49.572911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:49.572911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:49.572911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:49.575403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 395/783] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:49.576891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956bd5-5822-46f8-8fb7-ad609f3e40cd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.577847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956bd5-5822-46f8-8fb7-ad609f3e40cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.578278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956bd5-5822-46f8-8fb7-ad609f3e40cd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.580588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 396/784] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:49.581864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956bd5-5822-46f8-8fb7-ad609f3e40cd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.599417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.600021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.600305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.600580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.601762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] RBAC: Adding query filter params (interface=public, service_id=28a5fbc2f65c4114a6fde328cf98726d, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:49.602083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.604898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887405a-18a5-4fd3-97e4-88170238690d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.614220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 389/785] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/endpoints?service_id=28a5fbc2f65c4114a6fde328cf98726d&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:49.615465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.615756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.616034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.633476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:49.634727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:49.711051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956bd5-5822-46f8-8fb7-ad609f3e40cd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.723515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 390/786] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:49.725412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.759412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.759412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.759412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.759412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.759891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:49.759975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.761358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.775468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 397/787] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:49.776341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:49.798433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.801281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:49.855688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98810991-c599-4a06-885f-0f57877ff3b0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.893733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:49.894731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:49.895165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:49.895631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:49.896601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:49.898233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:49.900263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaebe-ff3f-4dd8-8a3b-ce679e30b50d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:49.905760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 391/788] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:49.906766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.000603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.001686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.002147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.002573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.027043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:50.048754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:50.049975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:50.050990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:50.060162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 398/789] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:50.066459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.108776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02614a7-6a84-4233-be3a-8704e264e41f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.135514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.179844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.181246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.181828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.182156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.192946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.194400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:50.216578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:50.217669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:50.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:50.220595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.222479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 392/790] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:05:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:50.223732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.245401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdeca2-9a6f-4285-8424-f6459c308e0d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.286902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.287632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.287917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.288194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.289207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:50.289515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.291501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.301372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 399/791] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:50.307368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.414470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224d4af-a685-4b88-9856-89f2bab6222c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.457095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.458984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.459612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.460852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.479624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.482037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.485664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.491948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf722fc-af2e-40ca-a5b4-895b70a794e0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:50.520411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:50.521564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:50.522686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:50.522686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:50.522686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:50.522686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:50.524244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 400/792] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:50.530022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.550058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.550847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.551128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.551421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.552272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:50.552637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.554228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.583493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 393/793] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:50.589555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.622221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.693347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18ad1-d5e2-473a-8499-56d4453dbb0b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.694764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b8184-5677-464c-a605-52d2bb216b47 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.730789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.731614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.731899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.732165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.743111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.744981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.761320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.762185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.762560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.762930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:50.763751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:50.764892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:50.767501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:50.767501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:50.767501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:50.767501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:50.768037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:50.768037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.768037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.771055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 394/794] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:50.772246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.794219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 401/795] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:50.799317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:50.871585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.884407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921cbb-95d2-4de5-b4c4-9bbdd6e416ab None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:50.921486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:50.922207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:50.922491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:50.922817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:50.923584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:50.923900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:50.925386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:50.942443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:50.943132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 395/796] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:50.944815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.040609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff25138-9ab3-47d0-b693-9d36e695cf32 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.062381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946bf8f-2d26-4b5b-840d-e4a99d694bb8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.086562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.088511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.088799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.089076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.099222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.100699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.101230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.102021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.102299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.102800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.111002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.112554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-05be79a3-7b7e-4309-8c44-b8b4f14420c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:51.119952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:51.120854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:51.121781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 402/797] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:51.123989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7d78-06b1-4205-8130-3edd2504fe8d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.123989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7d78-06b1-4205-8130-3edd2504fe8d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.123989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d7d78-06b1-4205-8130-3edd2504fe8d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.131260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 403/798] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:51.132295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d7d78-06b1-4205-8130-3edd2504fe8d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f3a829ca-54a3-491b-b6e3-91f5101ff7c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:51.133428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:51.134984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:51.135890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:51.139308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 396/799] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:51.141727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.142017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.143619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.150707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.164003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:51.165764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:51.259444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 397/800] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:51.260681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.280784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d7d78-06b1-4205-8130-3edd2504fe8d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.303723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.327717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.328499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.328881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.329169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.330078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:51.330577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.332025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.357312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 404/801] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:51.358312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.376172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47f915-293d-4a83-bd85-09f59b659e93 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.421889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.422979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.423472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.423931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.425536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:51.426106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.428423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.431641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.449406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 398/802] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:51.449406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.473960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b93ab7-5421-475f-aad9-7bec99840852 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.510858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.511607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.511873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.512142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.523887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2b8dfbd7-6421-47ed-b60b-91df8d5db565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:51.533514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:51.534474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:51.535915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 405/803] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:51.536619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afad93-1a49-4190-b2ea-bbdb72b0fcf6 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.536619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afad93-1a49-4190-b2ea-bbdb72b0fcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.536619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afad93-1a49-4190-b2ea-bbdb72b0fcf6 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.542229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 406/804] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:51.542229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afad93-1a49-4190-b2ea-bbdb72b0fcf6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.562476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127ef87-ad4c-462f-9061-e89213eaa2cb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.578504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.599297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.600090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.600377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.600653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.612316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.613850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:51.630913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:51.631853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:51.632748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:51.632748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:51.632748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:51.632748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:51.637523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 399/805] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:51.638370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.658028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afad93-1a49-4190-b2ea-bbdb72b0fcf6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.687110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.700610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.701507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.701781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.702059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/41c0ab6341464ac388e96852bb400679/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.726909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=41c0ab6341464ac388e96852bb400679, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.728572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cadd51-d5ef-4251-bea2-0a717dc123ba None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.744417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 407/806] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:51 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/41c0ab6341464ac388e96852bb400679/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:51.745516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dee91-b28d-4aaa-ac65-120767994fe6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:51.745726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.746014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.746297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.773064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:51.775055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:51.807126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:51.808475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:51.809431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:51.810195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:51.811475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:51.812080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:51.813862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:51.835627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 400/807] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:51.836738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.867058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 408/808] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:51.869354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:51.884274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:51.992275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fe824-e478-4679-87fd-7f860d65595a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.006280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f322f98-787a-44a9-957a-9b14cf65c8de None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.015868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.037701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.037701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.037701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.037701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.038084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.038084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.038084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.046789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 401/809] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.047687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.062501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.063789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.064318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.065042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.077430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.079258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:52.100128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:52.101146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:52.102044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:52.102044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:52.102044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:52.102044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:52.110034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 409/810] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:52.111433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.164322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8d996-08b1-430e-af09-1f4b79d0090f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.180659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.203623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.204423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.204709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.204987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] PATH_INFO: `/v3/services/network` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.205752 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.207250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:52.219706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:52.220686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 03:05:52.221948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:52.225972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 402/811] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:52.226853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.232547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e387d-8d6e-4ec0-9788-b756a4f3d7b4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.261193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.276097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.277255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.277838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.278418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.286166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.288058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:52.304769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:52.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:52.307593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:52.310091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 410/812] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:52.311085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.352948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1431cb9-d1d4-458e-b6c1-7d712d385663 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.394153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.394983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.395093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.395387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.396118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.396428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.397821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.415279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 403/813] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.415279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.437373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095851c-9022-4714-b12d-759f57c0cdc7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.461600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.476323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.477469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.478242 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.478782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.479753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] RBAC: Adding query filter params (name=network) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.480290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.481859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.487389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 411/814] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.488543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.514094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.525033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddebf5-14ae-4a7d-9287-13a52f8c5bb9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.563296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.563898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.564162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.564440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.565224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.565536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.567290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.579694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 404/815] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.584637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.598826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2811-f132-4507-b1a4-452775525511 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.635097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.635921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.636201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.636471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.644454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.645893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-78c6f99e-948d-4a50-9769-dfa8545caf94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:52.659897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:52.661054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:05:52.662168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:52.665843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 412/816] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:52.667117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9385f-6998-4cc9-8f24-e75bc606eeb3 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.667440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9385f-6998-4cc9-8f24-e75bc606eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.667703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9385f-6998-4cc9-8f24-e75bc606eeb3 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.668982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 413/817] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:52.669649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9385f-6998-4cc9-8f24-e75bc606eeb3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.702456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b4c6e-0993-4d72-a195-cbd01935d70b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.722754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.742091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.742760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.743170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.743481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.744239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.744629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.746085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.755565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 405/818] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.756514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.776714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:52.782673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9385f-6998-4cc9-8f24-e75bc606eeb3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.817739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.819234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.819605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.820182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/e6eb06c62ee340a0b70fbb88e9f3d09b/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.823121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=e6eb06c62ee340a0b70fbb88e9f3d09b, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.824783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574deff5-3adb-4274-a045-7d3063e497e5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.851278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 414/819] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:05:52 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/e6eb06c62ee340a0b70fbb88e9f3d09b/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:05:52.851278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.851278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.851278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.866380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c171c-6230-4c54-8dff-eb7842a400e9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:52.874776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:52.876524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:52.911450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:52.912586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:52.913183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:52.913520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:52.914459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:52.914977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:52.916781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:52.937953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 406/820] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:52.938961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:52.973460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 415/821] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:52.974772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.034006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.047711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.077557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3112f6-d7c8-43e3-bd77-d9aee4ce4503 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.126170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.126870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.127226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.127528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.128383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.129769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.130683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289fe13-a1c2-4522-969c-ac0c4fb0da49 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.155220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 407/822] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:05:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:53.156144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.200127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.200867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.201161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.201494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.276228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4798-48e6-4a0a-a8ab-6f4a0fb3b913 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.308424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.309921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.310433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.316304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.317139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.317429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.317712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.318385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.319935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:53.334077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:53.336161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:53.338654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 408/823] 10.222.0.39 () {56 vars in 1129 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:53.346180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.347847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.479329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 416/824] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:05:53.485161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.496751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9d28-2fa2-4439-b253-fbaee4b09d4a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.542975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.543677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.543966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.544236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.545485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:53.545794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.547841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.554534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 409/825] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:53.561401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.603265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce462ae3-8a09-4e39-b2d0-5286f33a6dfa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.635792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.648196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.655963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.656764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.657040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.657310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.671711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.674095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.686052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c55d2-e69f-46f1-835e-79bb48a01305 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:53.697728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:53.698811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:53.700789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:53.700789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:53.700789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:53.700789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:53.700789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 417/826] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:53.704250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.724250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.724926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.725210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.725485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.726195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:53.726607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.728660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.734198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 410/827] 10.222.0.39 () {56 vars in 1124 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:53.739747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.830339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf86c3-fb54-4bfb-afe6-ebd4070289a8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.865057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d915131-4c8d-458e-b715-c4400e65ab61 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:53.866742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.868435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.868786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.869134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.869943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:53.870322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.871804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.887893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 418/828] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:53.890621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:53.893365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:53.904883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:53.905966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:53.906251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:53.906519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:53.907280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:53.908981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:53.938795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 411/829] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:05:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:05:53.958700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:54.025420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e11ef-052b-425b-afaf-c8c11c7fafc8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.069221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.070023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.070300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.070568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.078632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:54.080195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:54.097212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:54.098208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:54.099177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 419/830] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:54.104724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:54.216705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616626b2-89dc-4748-9a76-95131e527ce6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.252270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.253179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.253631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.254048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.254964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:54.255437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:54.256991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1fdc-0da1-48fa-85c5-b47a39efe8d0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:54.270444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 412/831] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:54.276591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:54.291463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63fb12-1973-4ebb-aa68-4822518c3001 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.291997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63fb12-1973-4ebb-aa68-4822518c3001 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.292420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63fb12-1973-4ebb-aa68-4822518c3001 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.294203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 413/832] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:05:54.301401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.301854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.302417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.324451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:54.326535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:54.362234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:54.393866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:54.397125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3002f2-b8a5-48a1-8d8a-ba161807ffc3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.411281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 414/833] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:54.433113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:54.445498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.446627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.447142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.447688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.448885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:54.449430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:54.451043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:54.457801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 420/834] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:54.548345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f3579-f642-4df5-8021-ab4dc699b2d6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.589221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.590049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.590333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.590610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.603519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:54.613224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:54.614234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:05:54.615086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:54.618583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 415/835] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:54.628394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:54.736073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e1087-146f-4c32-8a6e-39ae568d7d8a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.771842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.772513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.772793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.773065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.773820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:54.774178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:54.775587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:54.784358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 421/836] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:54.790475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:54.820489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:54.900909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348a2e-6de6-4dd7-a0fb-deaaac796438 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:54.936609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:54.937414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:54.937696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:54.937973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:54.949789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:54.951274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:54.969506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:54.970427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:54.971587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:54.971587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:54.971587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:54.971587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:54.971587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 416/837] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:54.977922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.007902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.092369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580e8a2-4702-4c09-83d8-0efab1a250f4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.128479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.129150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.129426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.129858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.130588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:55.130887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.132280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2e264-6b11-4a3e-88e1-370121121e1f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.146037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 422/838] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:55.152591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.243849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.250845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c504d-0351-4856-a8b3-343d643a3f58 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.251136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c504d-0351-4856-a8b3-343d643a3f58 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.251428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c504d-0351-4856-a8b3-343d643a3f58 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.252776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 423/839] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:55.258040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.259339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.259884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.266313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a396b34-b5e1-4973-90fb-8d3e031cf4b5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.279643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:55.281369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:55.303546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.304363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.304642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.304919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.319271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.319271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:55.331898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:55.332881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:55.333810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 417/840] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:55.339361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.365907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 424/841] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:55.389900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.409909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.455608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59df12-2831-4094-81d7-63507c6dd1c6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.495935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.496764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.497149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.497524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.498552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:55.498965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.500629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.512308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.520651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd08f19-eefc-4fae-971f-3a7a54091188 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.526516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 418/842] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:55.527477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.560630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.561819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.562373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.562867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.574119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:55.582343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:55.583281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:05:55.584106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:55.591617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 425/843] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:05:55.593713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.636657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f88c1-ad74-4863-ba45-3f3299b9362c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.675309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.676017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.676300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.676576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.681089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.707624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdd6d-7c3e-4213-b689-43d321b98e7b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.745169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.746291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.746784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.747285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.748347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:55.748897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.750482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.759351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.766411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.766411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.767428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 426/844] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:55.768063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.877647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c2da-1e54-4185-814d-5009dbda9d27 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:55.895396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 419/845] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:05:55.899622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:55.925183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:55.926178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:55.926469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:55.926747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:55.937133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:55.942310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:55.944597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:55.966687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:55.967651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:55.968527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:55.968527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:55.968527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:55.968527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:55.970430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 427/846] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:55.976067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.016729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.021785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774db2a-6cf5-4771-a530-ae13fa850f2c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.059260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.060401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.060880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.061348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.073592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.075466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:56.095050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:56.096128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:56.097039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:56.097039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:56.097039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:56.097039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:56.099276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 420/847] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:56.105119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.105359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94039c9-0548-406f-b168-d7aa2127e703 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.142979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.143989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.144467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.144915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.145966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:56.146492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.148091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.169365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 428/848] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:56.174867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.236135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db145fb-dd39-4fab-a7a3-afaa994b6870 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.242534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.286935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.287907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.288197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.288473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.290195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:56.290516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.292252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.311403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 421/849] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:56.313859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.316566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3d19-48b7-47e1-a620-66bdc0dbca6d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.329253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.360133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.360925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.361204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.361503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.370783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.372879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:05:56.391771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:56.392815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 03:05:56.393833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:56.395332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 429/850] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:56.398891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.449966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc7261-914e-4afc-ac83-8ecc1cc369c3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.484884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.485659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.485942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.486218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.493218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.494653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a75da886-08e0-446b-b363-c9308ff5e4f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:56.508860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:56.509854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:56.511373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 422/851] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:56.514950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4aca9-91de-4511-8d23-e929d084de00 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.515517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4aca9-91de-4511-8d23-e929d084de00 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.515801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4aca9-91de-4511-8d23-e929d084de00 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.517226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 423/852] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:56.517844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4aca9-91de-4511-8d23-e929d084de00 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.521459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868fd7-8e71-4692-8961-b865dc69ca57 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.532748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.558079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.558743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.559022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.559955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.560158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:56.560458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.561885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4602f-d915-4285-9923-c31cd35c6b17 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.580088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 430/853] 10.222.0.39 () {56 vars in 1135 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:56.581721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.582204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.582664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.602947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:56.603817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.604271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:56.641617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4aca9-91de-4511-8d23-e929d084de00 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.676841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.677423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.677708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.677988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.678771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:56.679081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.681206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.683384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 431/854] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:56.684355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.698643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 424/855] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:56.705026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.788367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.793037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d45d9-9780-4738-9afe-a8075bdfbfd9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.816532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998d085-82c9-4444-9006-7cf9cf6ba6fc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:56.827535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.828176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.828454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.828724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.843982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:56.850086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:56.850743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:56.851023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:56.851424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:56.852864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=41c0ab6341464ac388e96852bb400679) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:56.853055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.854513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:56.860313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 425/856] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/role_assignments?user.id=41c0ab6341464ac388e96852bb400679&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:56.861083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:56.898968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:56.900834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d13e9a-4875-42e8-a0ea-628d4ebd7b9a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.002581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a537d-00aa-470a-96e9-f8e981be0081 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.019539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.027966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 432/857] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:05:57.033888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f323-b0db-426d-b72f-5e9f67af20f5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.034171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f323-b0db-426d-b72f-5e9f67af20f5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.034449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f323-b0db-426d-b72f-5e9f67af20f5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.036026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 433/858] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:57.036668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549f323-b0db-426d-b72f-5e9f67af20f5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.045185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.046102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.046738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.046738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.047335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.048722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8f10a3c3-8eee-47d6-b2fc-23a51e5446ce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:57.061370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:57.062431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:05:57.064544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 426/859] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:57.066185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.066185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.066185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.086785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:57.088043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:57.153804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.168815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 427/860] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:05:57.169865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.178593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549f323-b0db-426d-b72f-5e9f67af20f5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.223791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.224582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.225124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.225124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.235887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.237695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:57.256264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:57.257166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 434/861] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:57.258166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.280058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b10380-28e6-4303-8eeb-97c825b25fe4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.315344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.317483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.318155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.318437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.319231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.319424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.319730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.321913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.331637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 428/862] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:57.331637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.380359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d959da-b744-4717-9612-f4719315bf31 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.400907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.422715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.423568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.423850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.424125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.425201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.426947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:57.440452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:57.441429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:57.442267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 435/863] 10.222.0.39 () {56 vars in 1132 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:57.443105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.455550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb725d-1728-4717-a7d1-02fee86716e3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.490233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.490929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.491210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.491518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.492235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.492543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.494607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.505907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 429/864] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:57.506770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.559552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cda44-5346-46ee-93a7-ee4044c848e6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.574597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.597317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.598294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.598757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.599259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.600256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.600791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.602851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.608138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 436/865] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:57.609130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.622736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349c8c5-80c4-4833-b0c9-34991ef1ce45 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.644294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.670165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.671156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.671559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.671911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.672722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.673103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.674616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.685509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 430/866] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:57.686997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.756626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12285c71-1977-4b43-9aac-2e7d9b8fad99 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.797659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.798391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.798670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.798946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.804814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38e3e3-2b01-4869-b5f7-2737685ef74f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.807045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.809374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:05:57.825561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:57.826497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:57.827431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 437/867] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:57.832564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:57.859603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.860336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.860619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.860893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.861809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.862116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.864144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:57.871539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 431/868] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:57.882658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:57.953053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c006-3fde-4ba3-aa60-ed6b728ca4f3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:57.992116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:57.993791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:57.994360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:57.994840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:57.995911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:57.996429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:57.998177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7ef27-5fa6-4175-8f32-5205fefc3ab4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:58.012379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 438/869] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:05:58.023527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:58.068461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:58.142778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9b70e-404f-42b6-9b62-6bd4e79a16b7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:58.181618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:58.182666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:58.183161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:58.183758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:58.184811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=e6eb06c62ee340a0b70fbb88e9f3d09b) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:58.185335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:58.186983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ccb3-89b3-45a3-83db-09acec8361c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:58.193859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 432/870] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:05:58 2022] GET /identity/v3/role_assignments?user.id=e6eb06c62ee340a0b70fbb88e9f3d09b&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:59.209169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f106d-8af2-4ac7-a2d8-931a0807bf19 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.209169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f106d-8af2-4ac7-a2d8-931a0807bf19 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.209169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f106d-8af2-4ac7-a2d8-931a0807bf19 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.210218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 439/871] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:59.216247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.216639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.216844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.234709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:59.235902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:59.305480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 433/872] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:05:59.328769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f106d-8af2-4ac7-a2d8-931a0807bf19 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:59.437641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:59.443474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f106d-8af2-4ac7-a2d8-931a0807bf19 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:59.478707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:59.479862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.480330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.480784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.492256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:05:59.501266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:59.502336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:05:59.503215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:59.505624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 440/873] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:05:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:59.511643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:59.546196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:59.614758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692cf7a-5b8c-4771-b578-dc0984a5528c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:59.650746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:59.651731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.652167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.652590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.653487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:05:59.653948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:59.655495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:59.664826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 434/874] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:05:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:05:59.670488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:59.780316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dbc53-5d67-4af6-81bb-86c771f62964 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:59.816960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:05:59.817745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.818034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.818308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.829332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:05:59.830837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-04882d60-ea2b-4f7a-89cd-c9a50542dca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:05:59.849516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:05:59.850453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:05:59.851378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:05:59.851378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:05:59.851378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:05:59.851378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:05:59.851378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 441/875] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:05:59.858351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:05:59.958632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:05:59.959731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e199b-6b40-4ab4-abeb-3e4833560bed None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.960137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e199b-6b40-4ab4-abeb-3e4833560bed None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.960502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e199b-6b40-4ab4-abeb-3e4833560bed None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.966615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 442/876] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:05:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:05:59.967468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1161a74-6830-4f85-88c1-84ea9b5dacb8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:05:59.968561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:05:59.968848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:05:59.969122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:05:59.989911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:05:59.991127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:05:59.995542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.009924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.010827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.011386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.011825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.012810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:00.013386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.014756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.031009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 435/877] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:00.036258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.083563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 443/878] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:05:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:00.106575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.157787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e467c4-a819-42b3-af25-523c6b17a9fc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.198917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.199778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.200065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.200341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.208391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.209803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.214203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edebad-7801-4de5-aab9-d4fae543dfa5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-5934d44e-156d-4432-b3d1-d766e9f8304b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:00.224849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:00.226245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:00.227345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 436/879] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:06:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:00.227968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408701-2603-4655-92be-d14c0480e97a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.227968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408701-2603-4655-92be-d14c0480e97a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.228110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53408701-2603-4655-92be-d14c0480e97a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.229560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 437/880] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:00.231913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53408701-2603-4655-92be-d14c0480e97a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.251450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.257765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.258574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.258875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.259133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.259907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.261287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b6420-f310-4dd3-8841-d2d6492e2c30 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.270532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.281513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 444/881] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:00 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:00.281513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.281513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.281513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.309326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:00.310984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:00.393974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53408701-2603-4655-92be-d14c0480e97a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.411318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 445/882] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:00.438277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.441408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.442103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.442383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.442704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.443727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:00.443942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.445709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.470796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 438/883] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:06:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:00.476839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.570351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.576779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cc95d-47a3-4dbe-b750-a9db32c4c12c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.609169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69b5fc-bc25-4338-9916-610fca789ee7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.611163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.618084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.618751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.619034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.619769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.620514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.622239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e361220-0339-4db2-a915-c80b96665156 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.638667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 446/884] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:00 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:00.640586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe13c-bad6-4455-a29a-c6ef51332869 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.640942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe13c-bad6-4455-a29a-c6ef51332869 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.641295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe13c-bad6-4455-a29a-c6ef51332869 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.643291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 447/885] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:00.648733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.649871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.650408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.664575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.665391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.665743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.666096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.673054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:00.674622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:00.757477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.760337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:00.792833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 448/886] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:00.823172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.853459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.896774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 439/887] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:00 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 03:06:00.904314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:00.948063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2785fa-a903-4d56-95fb-a66364491f07 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:00.971639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:00.986409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:00.987212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:00.987617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:00.987903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:00.988561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:00.989943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:01.003734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:01.004836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:01.005690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 449/888] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:01.012661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.016591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20f873-0b8e-49f3-b8a5-6ea4c67afdc6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.060295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.062009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.062297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.062577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.077804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.079202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:01.103525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:01.105638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:01.107501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:01.107501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:01.107501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:01.107501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:01.110556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 440/889] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:01.111611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.141564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfe8ef-e250-4545-956e-cacb4cdd9233 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.143343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:01.179569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.181020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.181350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.181633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.182528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:01.182840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.185141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.195107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 450/890] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:01.197064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.251483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d098-f4f0-4758-a0f3-98b1cc382aec None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.252467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:01.299075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.299786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.300071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.300346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.301063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:01.301371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.302746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.319713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28c862-f774-47a3-b355-4dca6b8b20cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.327576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 441/891] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:01.329554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.358714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.359442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.359726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.360000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.360712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:01.361011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.362382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.370158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 451/892] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:01.436716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:01.464011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893af5e-c4a2-4f9e-83fa-0a54be325766 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.504199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.505014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.505297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.505570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.515545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.517139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:01.533184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:01.534170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:01.535056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:01.539961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 442/893] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:01.541405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.648622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bba4e-8e03-4aa6-91b1-80553e5dde61 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.662653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:01.685305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.685979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.686262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.686541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.687361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:01.687675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.689128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2ba7a-e36d-4f5c-951c-2b3abc812fe4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.707095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 452/894] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:01.708475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:01.824670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d90ce-886f-4d26-9fce-78e73b3cc303 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:01.860951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:01.861980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:01.862465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:01.862948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:01.864020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] RBAC: Adding query filter params (role.id=61570e75122a4b788521b019751d7dc3, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=41c0ab6341464ac388e96852bb400679) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:01.864547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:01.866199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74ec59-8bf8-4b08-9e04-e2ae05eb47cb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:01.872883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 443/895] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:01 2022] GET /identity/v3/role_assignments?user.id=41c0ab6341464ac388e96852bb400679&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=61570e75122a4b788521b019751d7dc3 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:02.637674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c27bc-b8ba-4e6e-bc53-a3f9e1884e06 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:02.638675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c27bc-b8ba-4e6e-bc53-a3f9e1884e06 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:02.639165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c27bc-b8ba-4e6e-bc53-a3f9e1884e06 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:02.641329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 453/896] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:02.646764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:02.647695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:02.648175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:02.654056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:02.667431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:02.668964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:02.739349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 444/897] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:02.762698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c27bc-b8ba-4e6e-bc53-a3f9e1884e06 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:02.874527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c27bc-b8ba-4e6e-bc53-a3f9e1884e06 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:02.910757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:02.911533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:02.911817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:02.912092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:02.912744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:02.914121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-2f631c40-b427-4629-99cf-da14b521fbcf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:02.926190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:02.927191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:02.928051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 454/898] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:02.935448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.022038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25efa-74c0-436b-bb18-a527527f0a3d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.022407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25efa-74c0-436b-bb18-a527527f0a3d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.022588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25efa-74c0-436b-bb18-a527527f0a3d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.025236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 455/899] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:03.033538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.033994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.034375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.050307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9721d9-8081-4f75-8277-47888710bbde None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.052474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:03.053788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:03.082827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.084531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.090948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.091881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.092296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.092710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.094263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.094734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.096281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.102063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 445/900] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:03.109252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.130993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 456/901] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:03.159325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.235306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61912-aaaf-46ec-ab4b-b85771219bea None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.274366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.275434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.275729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.276006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.276715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.277022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.278825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.287319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 446/902] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:03.291532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.296266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df25c8-dbf4-4999-ac9a-6fc7f31c8a81 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.341505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.342571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.342880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.343617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.344545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.346253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.357174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:03.363457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:03.364395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:03.365202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:03.365979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 457/903] 10.222.0.39 () {56 vars in 1132 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:03.373193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.409961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e676858-5582-484b-9242-25215bac9da0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.417129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.455431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.456751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.457193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.458122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.459442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] RBAC: Adding query filter params (interface=public, service_id=5965d67321da48faaa72d48a8840f4df, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.459914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.462674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.475040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 447/904] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/endpoints?service_id=5965d67321da48faaa72d48a8840f4df&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:03.478434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.511062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe58-3a18-4216-990e-8e304dd99536 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.549082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.550187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.550673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.551166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.552235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.552851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.554485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.560662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 458/905] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:03.569257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.605184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a4d7-ebd9-4ee7-b0ca-8b8b2c699818 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.631472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.643973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.644637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.644921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.645195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.645866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.647225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51946a7-b4bf-4beb-abc6-4729c7c10f6a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 448/906] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656068-6926-4691-8f35-43f85203cd34 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656068-6926-4691-8f35-43f85203cd34 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656068-6926-4691-8f35-43f85203cd34 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 449/907] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.663278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.663845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.685563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:03.686913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:03.692744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.705075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a6b9b-049c-47b8-9f0d-8b1a39b465b0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.746014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.747268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.747888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.748233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.749034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.749415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.751049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.762739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 459/908] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:03.764176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.779917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 450/909] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:03.803852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.892800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71362b92-8942-449d-adc4-391ab3de1a63 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.934733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.935957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.936637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.937083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.938791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] RBAC: Adding query filter params (interface=public, service_id=a2b4697d549e4c00a2ef3340f414824c, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:03.939289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.941687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:03.943279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12400f7-5b3a-4dd3-8178-b40fb04e76ef None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:03.951488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 460/910] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/endpoints?service_id=a2b4697d549e4c00a2ef3340f414824c&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:03.957812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:03.969738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.989183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:03.991148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:03.991797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:03.992114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:03.992947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:03.993652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:03.995088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28676cc2-1664-4e6b-bbad-805241baa4e7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:04.019685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 451/911] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:03 2022] POST /identity/v3/services => generated 241 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:04.100114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14ef88-a229-4ade-b55c-295c96dceecf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:04.149621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:04.150685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.151226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.151745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.152762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:04.154400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701de0a-429d-4f5d-b48b-adff1aa579ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:04.167355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 461/912] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:03 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:06:04.188879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7dee7-673a-4727-954b-aa0319268012 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.189565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7dee7-673a-4727-954b-aa0319268012 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.190061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7dee7-673a-4727-954b-aa0319268012 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.192220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 452/913] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:04.197986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.198308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.198543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.217550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:04.219160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:04.299475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 462/914] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:04.322279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7dee7-673a-4727-954b-aa0319268012 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:04.433355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7dee7-673a-4727-954b-aa0319268012 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:04.468630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:04.469440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.469713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.470029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.481439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:04.489624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:04.490552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:04.491851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 453/915] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:06:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:04.497256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:04.512509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:04.610161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673606-0dee-44d1-a77e-ff15358e6d2a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:04.645800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:04.646479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.646759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.647033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.647813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:04.648120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:04.650048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:04.658544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 463/916] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:06:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:04.663888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:04.705813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:04.772353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290a803-1b1a-4473-8256-ccdf76a926c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:04.810507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:04.811579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:04.812011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:04.812462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:04.823529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:04.825210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:04.844402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:04.845534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:04.846631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:04.846631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:04.846631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:04.846631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:04.848601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 454/917] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:04.859273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:04.941660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:04.969673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5129fc1-3843-4daf-a10e-cdc53722aee7 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.005308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.006333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.006810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.007318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.008349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:05.008895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.010471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.023176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 464/918] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:05.028943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.125633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:05.137606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b6d48-e45d-420d-ac7f-b7f46fd8a6bb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.179902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.181276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.181799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.182329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.189738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.191408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:05.207316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:05.208254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:05.209222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:05.212550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 455/919] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:05.218323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.322947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240676df-ecc2-497a-a6d1-8dc519c7ec67 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.358492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.359579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.360059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.360525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.361553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:05.362076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.363754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.374546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:05.383767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 465/920] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:05.389558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.499779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882dd03-0e43-442e-8bfa-cc6490a115c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.535364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.536160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.536440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.536715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.547812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.549283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:05.566859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:05.567832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:05.568715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:05.568715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:05.568715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:05.568715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:05.568715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 456/921] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:05.574327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.676323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d20d82-ba9a-4815-8599-b33660f0a362 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.711521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.712181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.712457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.712726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.713446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:05.713748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.715219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.727667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 466/922] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:05.733189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.759676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:05.846848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c18eb-99d7-407b-a673-0069fcbb7e16 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:05.881957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:05.882754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:05.883575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:05.883854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:05.891279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:05.892741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-b948eea8-bd04-4e0b-af8e-4bf2560343cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:05.906711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:05.907635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:05.908538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 457/923] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:05.913830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:05.969630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.018357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447e55-c860-4186-951d-7cbee1907f93 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.018643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447e55-c860-4186-951d-7cbee1907f93 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.018839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34447e55-c860-4186-951d-7cbee1907f93 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.019153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4825018-be2d-4279-b269-20d7c413a27c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.020495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 458/924] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:06.025772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.026063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.026341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.045725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:06.047037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:06.065269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.066304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.066584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.066854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.067671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:06.067976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.069932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.088998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 467/925] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:06.094892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.109168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.127368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 459/926] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:06.146061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.218608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dbc99-b6b0-48da-9d38-0209891e8293 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.233985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.259924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e24fe-6370-4ee7-a649-55eb4cb08df9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.260460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.261548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.261898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.262245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/41c0ab6341464ac388e96852bb400679/roles/61570e75122a4b788521b019751d7dc3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.264781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=41c0ab6341464ac388e96852bb400679, role_id=61570e75122a4b788521b019751d7dc3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.266512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.284927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 468/927] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:06:06 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/41c0ab6341464ac388e96852bb400679/roles/61570e75122a4b788521b019751d7dc3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:06:06.320024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.321604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.321934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.322266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.322942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.324771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a7d86ce5-d2fc-4bcc-a1ca-19368123a28a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:06.338384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:06.340785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.342839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 460/928] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:06.347357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.469818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a571c-aa62-4ca7-b435-5611996ac282 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.472622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f477652-e5c6-4701-8091-cdc2d388e952 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.473182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f477652-e5c6-4701-8091-cdc2d388e952 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.473667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f477652-e5c6-4701-8091-cdc2d388e952 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.475567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 461/929] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:06.481851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c763767-e5b0-48c9-8d13-b5a1f04fff57 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.482318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c763767-e5b0-48c9-8d13-b5a1f04fff57 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.482601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c763767-e5b0-48c9-8d13-b5a1f04fff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.509494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1c763767-e5b0-48c9-8d13-b5a1f04fff57 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:06.510695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c763767-e5b0-48c9-8d13-b5a1f04fff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:06.511030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.511727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.512011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.512287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.513268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:06.514422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.516158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.522387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 469/930] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:06.528147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.572984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.574722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.593308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 462/931] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:06.595119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017730d-aecb-4ca2-958c-560776c9d5a2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.595434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017730d-aecb-4ca2-958c-560776c9d5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.595715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017730d-aecb-4ca2-958c-560776c9d5a2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.596980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 463/932] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:06.602636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.602914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.603197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.621121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:06.622306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:06.652925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf326b8-427d-4e9c-9a27-e8102d6917a0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.689843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.690511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.690789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.691063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.691807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:06.692112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.693496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.693891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 464/933] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:06.695017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.712102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 470/934] 10.222.0.39 () {56 vars in 1114 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:06.712974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.802805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68256d24-9af0-4329-a04a-5b1b7b660caf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.823114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.826432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bae5-931b-4eb4-9834-7b69b06f8cd6 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:06.828803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:06.851362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21439964-ef63-4df1-8db1-78541258105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.852565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439964-ef63-4df1-8db1-78541258105e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.852848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439964-ef63-4df1-8db1-78541258105e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.853123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21439964-ef63-4df1-8db1-78541258105e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.853790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21439964-ef63-4df1-8db1-78541258105e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.855177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21439964-ef63-4df1-8db1-78541258105e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-21439964-ef63-4df1-8db1-78541258105e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:06.868164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:06.869345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:06.870170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 465/935] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:06.870955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21439964-ef63-4df1-8db1-78541258105e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:06.882739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:06.884092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:06.884380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:06.884652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:06.885396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:06.886776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:06.908631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 471/936] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:06.908631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.010949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21439964-ef63-4df1-8db1-78541258105e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.057517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5ea41-b166-4f7c-a5f6-9b8d9b4b6393 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.063196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.064130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.064420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.064700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.065392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.067549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:07.085126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:07.086172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:07.087048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 466/937] 10.222.0.39 () {56 vars in 1132 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:07.093759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.115925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.116691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.117048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.117943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.118748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:07.119403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.120639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.128453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 472/938] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:07.134691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.163588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:07.166482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:07.255741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881853dd-b724-4f17-b54c-b9ffa240eab3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.266162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2632f9-9dca-48d9-b9ea-2c6fa6026d8a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.292409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.293279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.293658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.294027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.294828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:07.295340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.296884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.302843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 467/939] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:07.307978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.308884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.309167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.309440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.310154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:07.310459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.311873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.311975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.317330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 473/940] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:07.323761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.431885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cb86-401d-4408-b1c1-c5ce1d78bf06 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.436767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:07.444009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:07.465147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb649b6-1882-4911-820d-03717398ae04 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.480494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.481173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.481489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.482226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.482946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:07.483271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.485324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.496500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 468/941] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:07.502290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.512368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.512368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.512368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.512368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.512939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] RBAC: Adding query filter params (interface=public, service_id=c9ea3ca39df3424ea5aee97f752e9613, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:07.513103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.514535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.528597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 474/942] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:06:07 2022] GET /identity/v3/endpoints?service_id=c9ea3ca39df3424ea5aee97f752e9613&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:07.536309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:07.618340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2b803-7cc8-4168-9ab2-25cbf1ac1e78 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.654459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.655422 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.655845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.656377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.657209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.658665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9799fd-8999-4cb7-94b3-a19d27c5bbeb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.660507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46923ab-0f6d-4853-ac14-35e6cba54b37 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:07.687291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 469/943] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:07.716410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:07.717145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:07.717427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:07.717697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:07.718397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:07.719818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc9468-af5a-4cba-80fe-18d4d34cbf83 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:07.727822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 475/944] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:07 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:06:08.585017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012e8-55dc-4e41-9a89-2280a7da3bb8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:08.585532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012e8-55dc-4e41-9a89-2280a7da3bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:08.585643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012e8-55dc-4e41-9a89-2280a7da3bb8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:08.587919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 470/945] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:08.593546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:08.593970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:08.594114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:08.601186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:08.612451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:08.613647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:08.703353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 476/946] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:08.729363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e012e8-55dc-4e41-9a89-2280a7da3bb8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:08.768671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:08.856014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e012e8-55dc-4e41-9a89-2280a7da3bb8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:08.920339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:08.921162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:08.927261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:08.927261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:08.936112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:08.952342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:08.953350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:08.954269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 471/947] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:06:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:08.959590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.158832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cd6f7-4eac-47ca-98db-ef56bb7cfb91 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.196295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.197099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.197377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.197648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.198409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:09.198711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.200122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40039068-9a06-4709-a88d-f549fe61ae34 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.201578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.209440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 477/948] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:06:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:09.214744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.313196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.325966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355dd1-8a3b-413b-b9fc-8ba616b5a606 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.327316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617919-7bf4-4591-82b5-18ba88111f56 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.327615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617919-7bf4-4591-82b5-18ba88111f56 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.327897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37617919-7bf4-4591-82b5-18ba88111f56 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.329399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 478/949] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:09.334141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.334431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.334709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.355384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:09.356657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:09.366228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.367866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.368333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.368787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.380697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.382361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:09.402045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:09.402952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:09.403881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:09.403881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:09.403881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:09.403881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:09.403881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 472/950] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:09.410532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.455640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 479/951] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:09.481875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.531939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbc0b-bde2-4b0c-9c11-7b1a5bb65014 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.578505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.579877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.580387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.581121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.582690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:09.583210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.584326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d789495-13e2-4a06-a02a-1d2962ed55c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.585098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.602271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.612550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 473/952] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:09.614054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.619386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.629868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.630530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.630805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.631073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.631816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.633977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.635597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING keystone.server.flask.common [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 03:06:09.658091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 03:06:09.658091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 03:06:09.658091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [parameters: {'id': 'ec129f1a3ada464d98860edce38b51da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 03:06:09.658091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec129f1a3ada464d98860edce38b51da.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec129f1a3ada464d98860edce38b51da. May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:06:09.666850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 03:06:09.667714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() May 26 03:06:09.668666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 03:06:09.669558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 03:06:09.670482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:06:09.671289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'id': 'ec129f1a3ada464d98860edce38b51da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.672286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:09.673111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec129f1a3ada464d98860edce38b51da. May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.674070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 480/953] 10.222.0.39 () {60 vars in 1140 bytes} [Thu May 26 03:06:09 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:09.676069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.758269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cced45-ed9b-49a2-b469-b4ba358cf08f None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.788847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4713-c6d3-4a8a-a414-d884d6d66f5a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:09.806553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.807416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.807696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.807964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.815845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.817241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.830244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:09.831423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:09.831929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:09.832404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:09.832940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:09.833834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 03:06:09.834815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:09.839718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 474/954] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:09.840586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.845380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:09.847347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:09.865826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:09.866726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:09.868247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:09.871557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 481/955] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:09.874997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:09.910205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.932704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:09.958129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23b6ab-7bf3-4469-8e56-b539d61cf378 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.005586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.006662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.006933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9064f67-305f-4626-94a7-b498222798ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.007435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.008038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.009586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:10.011199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.012932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.041173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 475/956] 10.222.0.39 () {56 vars in 1149 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:10.042187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.055797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.057515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.058101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.058562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.059850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:10.060394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.063003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bedcf8-9f48-4230-a319-0f855af9886e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.090780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 482/957] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:10.166265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de998511-ed2a-407a-8ed9-dc5459da95a5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.186909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.190513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91440af-b14a-4fa3-996d-962f28dd2c3b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.190812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91440af-b14a-4fa3-996d-962f28dd2c3b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.191102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91440af-b14a-4fa3-996d-962f28dd2c3b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.193174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 483/958] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:10.200936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3ec4-83e2-49e6-8be7-1bfff52820f7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.200936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3ec4-83e2-49e6-8be7-1bfff52820f7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.200936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3ec4-83e2-49e6-8be7-1bfff52820f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.212363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.213033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.213312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.213587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.219853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-190f3ec4-83e2-49e6-8be7-1bfff52820f7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:10.221829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-190f3ec4-83e2-49e6-8be7-1bfff52820f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:10.291686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.293317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.297510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 484/959] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:10.300701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab2de-15cc-45ab-959a-ddbb7ba15a6d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.301257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab2de-15cc-45ab-959a-ddbb7ba15a6d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.301815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecab2de-15cc-45ab-959a-ddbb7ba15a6d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.303375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 485/960] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:10.309090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.309385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.309669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.328419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:10.329618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:10.358176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.411068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 476/961] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:10.412221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.415345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 486/962] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:10.418028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.435797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.521823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6179-c873-49ef-9797-add63000a56d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.532664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd79f-9eff-40f4-8e20-fbf6c3ac4b9e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.557718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.558435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.558718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.558997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.559699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.561071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:10.573440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:10.574529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.575425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 477/963] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:10.576241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.578239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.579552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.580940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.581439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.595126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.597278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:10.618417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:10.619385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:10.620747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:10.620747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:10.620747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:10.620747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:10.620747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 487/964] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:10.621029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.638936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.666952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.691263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcb5c9-c00c-4ace-89d2-584905b91f25 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.728325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.729134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.729417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.729694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.730343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.731845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.736742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217897b-81e8-40f0-b86b-020b265d049b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:10.743867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:10.744881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:10.745718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 478/965] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:10.746513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.779730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.780391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.780672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.780948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.781646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:10.781957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.783349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.792898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 488/966] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:10.792898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.858043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba120ae7-13dc-4338-98b2-a2954079f5fb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.878513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.894563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.895233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.895549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.895825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.896542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:10.897072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.898216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b097e-5bc3-4707-93f5-b71f2d1f0b07 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:10.910270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 479/967] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:10.912052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:10.922798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:10.940869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:10.941525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:10.941807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:10.942078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:10.942773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:10.943075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:10.944470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:10.953737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 489/968] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:10.953737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:11.029006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0e29f-60b1-4244-945c-69aa786ed5a6 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.053846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e690d-7f0c-4bb8-be44-47249fd0c60c None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.065000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.065665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.065950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.066226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.066926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:11.067233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.068621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.083265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 480/969] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:11.083265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:11.090208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.091077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.091407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.091729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.100957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.103129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:11.118800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:11.119780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:11.120656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 490/970] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:11.126038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:11.129207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:11.202577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:11.233876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0acca-7beb-4a83-a1c1-114ad0c22382 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.255740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e9711-7242-4aeb-91e9-b8c4cd7c8c26 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.274228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.274228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.274228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.274631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.275194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:11.275636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.277262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.282815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 481/971] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:11.288476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:11.296399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.297070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.297359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.297636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.298429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:11.298744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.300360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.316454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 491/972] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:11.323374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:11.392565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:11.435178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de473e4-2a43-466d-a60d-55b8cbaaea6b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.437376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e721-5d00-4664-8135-8fa344091a13 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:11.474986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.475997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.476536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.477007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.478026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] RBAC: Adding query filter params (role.id=61570e75122a4b788521b019751d7dc3, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=41c0ab6341464ac388e96852bb400679) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:11.478526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.478834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:11.479594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:11.479966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:11.480310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:11.481009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9007c90-0b96-4d71-911b-0d24cd252310 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.481133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:11.482524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb565f-244e-439a-8882-46d0921e5d7e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:11.488146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:11.488466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 492/973] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:11 2022] GET /identity/v3/role_assignments?user.id=41c0ab6341464ac388e96852bb400679&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=61570e75122a4b788521b019751d7dc3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:11.510771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 482/974] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:12.462740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb652a-5cc2-4da2-9b73-609754a0dc02 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:12.463287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb652a-5cc2-4da2-9b73-609754a0dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:12.463287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb652a-5cc2-4da2-9b73-609754a0dc02 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:12.464905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 493/975] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:12.470515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:12.470962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:12.471044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:12.489028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:12.490207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:12.565818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 483/976] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:12.590404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb652a-5cc2-4da2-9b73-609754a0dc02 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:12.697119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb652a-5cc2-4da2-9b73-609754a0dc02 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:12.727673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:12.737942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:12.738934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:12.739479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:12.739976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:12.740875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:12.742916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:12.758395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.sql.core [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 03:06:12.758395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 03:06:12.758395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [parameters: {'user_id': 'eb1dc17f7c304fdba3c0c83c277807d9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 03:06:12.758395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80416) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:06:12.767042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:06:12.767966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application yield resource May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application next(self.gen) May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.rollback() May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.force_reraise() May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise self.value May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application session.commit() May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 03:06:12.768919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._prepare_impl() May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.session.flush() May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._flush(objects) May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application compat.raise_( May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application flush_context.execute() May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rec.execute(self) May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 03:06:12.769836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 03:06:12.770822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result = self._query(query) May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application conn.query(q) May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application result.read() May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 03:06:12.773299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [parameters: {'user_id': 'eb1dc17f7c304fdba3c0c83c277807d9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:12.775362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 03:06:12.777347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 03:06:12.779621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:12.782337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 494/977] 10.222.0.39 () {60 vars in 1131 bytes} [Thu May 26 03:06:12 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:12.788575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:12.900509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a80d7-a967-4d75-9919-fffe6f363dbe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:12.917528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:12.936627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:12.937433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:12.937719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:12.937989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:12.951995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:12.953444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-9f10c7c1-8da9-4af1-b52c-9c736d5c9cf9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:12.969063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:12.971309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:12.973712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 484/978] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:12.981763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:13.090392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd71a01-c9fc-4764-934a-f5175464afb1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:13.127717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:13.128811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.129299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.129779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.130817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:13.131366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:13.133495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a67d02-ab0a-47ac-aa6a-cf3448294a4a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:13.174350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 495/979] 10.222.0.39 () {56 vars in 1101 bytes} [Thu May 26 03:06:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:13.466066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8473d-a231-4865-bbda-afafe562d701 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.466416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8473d-a231-4865-bbda-afafe562d701 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.466625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8473d-a231-4865-bbda-afafe562d701 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.468339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 485/980] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:13.473847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a6f1-1b28-4c84-8595-469c53d2ace4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.474313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a6f1-1b28-4c84-8595-469c53d2ace4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.474408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036a6f1-1b28-4c84-8595-469c53d2ace4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.492844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a036a6f1-1b28-4c84-8595-469c53d2ace4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:13.494436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a036a6f1-1b28-4c84-8595-469c53d2ace4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:13.539068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:13.569898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 496/981] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:13.590216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8473d-a231-4865-bbda-afafe562d701 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:13.702386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8473d-a231-4865-bbda-afafe562d701 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:13.735963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:13.736629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.736911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.737185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.737912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:13.739303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b7620-f35b-43ec-a6b8-9b4859c1d0c1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:13.759274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 486/982] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:13 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:13.848872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46adc1e-1627-4e87-b26a-f468fe6dd0a5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.849241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46adc1e-1627-4e87-b26a-f468fe6dd0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.849436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46adc1e-1627-4e87-b26a-f468fe6dd0a5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.851001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 497/983] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:13.857005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77b9f-393e-4003-aa61-e72b0401799a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.857327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77b9f-393e-4003-aa61-e72b0401799a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.857431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.857597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb77b9f-393e-4003-aa61-e72b0401799a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.857692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.857907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.859098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 498/984] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:13.864435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:13.864724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:13.865003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:13.876296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:13.877502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:13.884841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:13.886633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:13.910042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:13.969075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 487/985] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:13.974679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 499/986] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:13.993695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:13.996731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.098174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949ab54-4c5d-4558-9bd9-7d87b3d41393 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.102092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5071c-a032-42bd-8385-bfc0586d3a0f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.103788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:14.134816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.136009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.136465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.136895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.137550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.138912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.141281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.142069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.142349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.142622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] PATH_INFO: `/v3/services/image` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.143306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.144946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:14.151921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:14.152917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:14.153785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:14.155613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 488/987] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:14.157176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:14.158107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:14.158945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:14.160599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.163332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 500/988] 10.222.0.39 () {56 vars in 1105 bytes} [Thu May 26 03:06:13 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:14.165391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.190250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:14.274089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72871fc-754f-4636-b3f7-d8eebb8d7230 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.274278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c415cff-c200-4fce-b413-801910a53804 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.310796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.311495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.311781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.312064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.312270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.312777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] RBAC: Adding query filter params (name=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:14.313082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.313402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.313843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.314260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.314457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.315282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:14.315749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.317277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.319648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 501/989] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:14.325250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.327272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 489/990] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:14.332461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.362191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:14.432574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:14.443926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c84dc-3974-4663-9739-1986556be9fc None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.444307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de6f8f-1df7-4b41-bd0a-bf11da7d3288 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.479775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.480300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.480745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.480745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.481001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.481109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.481278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.481500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.481655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:14.481942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.482201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] RBAC: Adding query filter params (type=image) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:14.482505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.483323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.483899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.488454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 490/991] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:14.494761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.495595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 502/992] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:14.606296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcf01e-b62b-4393-a2bb-438c94706f75 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.643179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.643893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.644172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.644444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.645251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] RBAC: Adding query filter params (interface=admin, service_id=c9ea3ca39df3424ea5aee97f752e9613, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:14.645553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.646944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf5cf5-7262-4c47-9611-eaf5a4b24cde None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.658011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 491/993] 10.222.0.39 () {56 vars in 1249 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/endpoints?service_id=c9ea3ca39df3424ea5aee97f752e9613&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:14.659535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:14.760509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:14.768439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee35f7-aa68-4222-ba89-b9a52965eb6e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:14.801970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:14.802631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:14.802916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:14.803191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:14.803896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:14.805422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607fb30-6186-4c70-8eea-7ae7124d0b80 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:14.811661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 503/994] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:14 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:06:16.045923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe2a8-ed7c-4e2c-acf3-513c532648f5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.046443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe2a8-ed7c-4e2c-acf3-513c532648f5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.046522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbe2a8-ed7c-4e2c-acf3-513c532648f5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.047897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 492/995] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:16.053096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.053477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.053656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.071975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:16.073171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:16.141395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 504/996] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:16.164649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe2a8-ed7c-4e2c-acf3-513c532648f5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.184352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:16.270761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbe2a8-ed7c-4e2c-acf3-513c532648f5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.305199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.305903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.306186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.306458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.307108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.308515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:16.320935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:16.322028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:16.322988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 493/997] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:16.329569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.430516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:16.443694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ecdfc-b015-4bd9-aa77-75c47dc89b40 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.478261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.478939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.479224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.479550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.480260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:16.480570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.481944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.493489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 505/998] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:16.493669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.581564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:16.602144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95962b-260d-4e69-99df-68c5c35b7a2d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.635764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.636422 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.636703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.636974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.637676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:16.637982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.639364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fe7d5-3a8d-41e9-93f1-ab2954f15fe5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.647599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 494/999] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:16.650806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.695878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7cac-de0d-46cb-abd6-15bb06f87a4e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.696253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7cac-de0d-46cb-abd6-15bb06f87a4e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.696455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7cac-de0d-46cb-abd6-15bb06f87a4e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.697786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 495/1000] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:16.702800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.703085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.703387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.720432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:16.721605 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:16.758931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c044b-4c40-4c01-af13-3117080f8c25 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.796204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.796864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.797153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.797429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.799325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 496/1001] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:16.799448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] RBAC: Adding query filter params (interface=public, service_id=f48f9121bc2947b9b964dc1d3a8b4448, region_id=RegionOne) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:16.799448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.800047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.807277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 506/1002] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:06:16 2022] GET /identity/v3/endpoints?service_id=f48f9121bc2947b9b964dc1d3a8b4448&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:16.812169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.821510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:16.840988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:16.906305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:16.935778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243bd5c-eb82-4f7d-aa0e-366d1feaaf45 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.937351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227a8e-e7bb-4da9-8f44-c7b1d99cb301 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:16.972780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.973448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.973732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.974007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.974548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:16.974730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.975212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:16.975534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:16.975793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:16.976085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68d32b-b7c4-4f02-a871-d988a6e9eddc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.976762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:16.978218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87570-ec41-45b9-a4c3-51d8d27154ce None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:16.983672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 497/1003] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:16 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:06:16.998493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 507/1004] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:16 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:17.067380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8932ae-6648-49c8-b829-8c1d8e477348 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.067927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8932ae-6648-49c8-b829-8c1d8e477348 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.067927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8932ae-6648-49c8-b829-8c1d8e477348 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.069708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 498/1005] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:17.075694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.076222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.076298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.095230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:17.096458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:17.184381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 508/1006] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:17 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:17.208052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8932ae-6648-49c8-b829-8c1d8e477348 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:17.317817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8932ae-6648-49c8-b829-8c1d8e477348 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:17.353200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:17.354339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.354860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.355368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.356211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:17.357777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:17.370433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:17.371667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:17.372525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 499/1007] 10.222.0.39 () {56 vars in 1126 bytes} [Thu May 26 03:06:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:17.379781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:17.382522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:17.484748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9cfd-d150-413e-b647-b3089cb34466 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:17.519692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:17.520371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.520653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.520929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.521643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:17.521952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:17.523956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:17.529160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 509/1008] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:17.534627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:17.539547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:17.640898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5607-941b-49bf-a090-2bd343da5eb0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:17.673763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:17.674428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.674710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.674985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.675720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:17.676027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:17.677399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25c2ab-99c2-4436-9821-85d6dabded4c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:17.683134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 500/1009] 10.222.0.39 () {56 vars in 1122 bytes} [Thu May 26 03:06:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:17.688590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:17.754194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:17.791872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017e1f-ed01-4003-b8c3-0627dc8bf3e5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:17.826673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:17.827366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:17.827651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:17.827929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:17.828673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:17.830050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482079c-62fc-4e3b-b67b-7858a4403215 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:17.845242 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 510/1010] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:19.231649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438c10-1c62-443f-9867-c58c71413b5d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.232402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438c10-1c62-443f-9867-c58c71413b5d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.232402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a438c10-1c62-443f-9867-c58c71413b5d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.233548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 501/1011] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:19.239012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.239474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.239662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.256815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:19.258059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:19.260383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ff0f-820a-4c58-ba60-52ebf5940d92 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.260678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ff0f-820a-4c58-ba60-52ebf5940d92 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.260958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19ff0f-820a-4c58-ba60-52ebf5940d92 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.262310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 502/1012] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:19.267316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.267607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.267884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.284201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:19.285371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:19.335763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 511/1013] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:19 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:19.354199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.358358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.364520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 503/1014] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:19 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:19.388077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.417364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.467612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af84ea9-3f2c-413d-93f6-21b456c8108e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.491666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf3178-4d72-4853-abca-055ea779e877 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.503987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.504783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.505066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.505342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.505993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.507916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:19.520567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:19.521593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:19.522426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 512/1015] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:19.527660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.530346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.531141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.531448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.531721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] PATH_INFO: `/v3/services/image` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.532374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.533737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:19.547004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:19.548212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:19.549222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 504/1016] 10.222.0.39 () {56 vars in 1105 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:19.554019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.619015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.631867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f943a8b-d389-455e-a7f7-6ea50e198218 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.648607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.662143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326ec7f-c539-4adc-b2ed-49f8ea2a26af None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.667153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.667840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.668119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.668391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.669098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:19.669407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.670774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.676246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 513/1017] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:19.682101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.702113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.702784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.703065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.703368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.704068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:19.704372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.705875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.715483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 505/1018] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:19.716489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.794718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc443ace-03be-47a3-b950-17b3370498bd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.822764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3e858-3b6c-48fd-b27b-9c20760547ae None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.831103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.831797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.832073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.832344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.833042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:19.833346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.834783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.840677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 514/1019] 10.222.0.39 () {56 vars in 1112 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:19.846751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.856277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.866173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.867519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.867855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.868125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.868829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] RBAC: Adding query filter params (type=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:19.869188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.870557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.880485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 506/1020] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:19.881227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:19.895935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:19.956340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bea5e-7c87-485a-9521-784fbe0b6580 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:19.992959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:19.993619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:19.993904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:19.994177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:19.994922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:19.996332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:19.997354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc84dc-66d2-4e83-8573-6b089a2e299c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:20.018099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 515/1021] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:06:20.058863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:20.059563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.059850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.060125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.060936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] RBAC: Adding query filter params (interface=public, service_id=4d995cbad2d34defadae3f75b7e7aa51, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:20.061250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:20.063044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29270fdd-0ef2-47ca-9c02-64a38c549354 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:20.074192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 507/1022] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:06:19 2022] GET /identity/v3/endpoints?service_id=4d995cbad2d34defadae3f75b7e7aa51&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:20.075524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:20.101695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d20db3-39f1-4a5c-a90b-6401e81f9901 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.102035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d20db3-39f1-4a5c-a90b-6401e81f9901 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.102281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d20db3-39f1-4a5c-a90b-6401e81f9901 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.103892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 508/1023] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:20.109615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.109875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.110073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.138119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:20.140413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:20.196408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:20.200910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54034-d090-4d7c-ae40-97b73b63c308 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:20.213162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:20.240943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 509/1024] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:20.250607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:20.251469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.252154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.252286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.253510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:20.255620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:20.265606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:20.268039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 516/1025] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:06:20 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:06:20.411053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9e41f-21d0-4f18-847e-17f6a3001caf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:20.451349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:20.452177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.452455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.452728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.453968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:20.455498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92294ed8-faca-4ddc-8bbc-7e8dc6e5c75b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:20.463107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 510/1026] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:20.464850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:20.595705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa42d-467f-463f-babb-a3d29bb16d57 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:20.633885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:20.634621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:20.634906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:20.635174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:20.635932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:20.637380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec984b01-4a0d-44ff-b996-acb265242d3b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:20.675271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 517/1027] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:20 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:22.493673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23964c02-a0f7-4591-b1dd-44b75c3575c5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.494186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23964c02-a0f7-4591-b1dd-44b75c3575c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.494264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23964c02-a0f7-4591-b1dd-44b75c3575c5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.495760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 511/1028] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:22.500861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.501284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.501478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.519527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:22.520721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:22.532555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:22.549646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e4e1-b481-425a-9cd2-be1acf36bc1d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.549952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e4e1-b481-425a-9cd2-be1acf36bc1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.550205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e4e1-b481-425a-9cd2-be1acf36bc1d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.551619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 512/1029] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:22.560164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.560164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.560164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.579176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:22.580436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:22.604125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 518/1030] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:22.627858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:22.631047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:22.660315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 513/1031] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:22.684134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:22.741116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaeb43-df8a-4f03-bf9b-b31459dd5adb None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:22.780168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:22.781001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.781284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.781560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.794130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:22.798110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999de6b-e79f-4142-bf95-2e55fe397b7e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:22.802374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:22.803332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:22.804857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:22.807680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 519/1032] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:06:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:22.809975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:22.819143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:22.836835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:22.837679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.837962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.838232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] PATH_INFO: `/v3/services/image` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.838868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:22.840257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:22.853033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:22.854031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:22.854867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 514/1033] 10.222.0.39 () {56 vars in 1105 bytes} [Thu May 26 03:06:22 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:22.860237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:22.921782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:22.924816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589a42b-b93e-4b64-9a15-ce1ab787a2ac None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:22.969811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:22.970478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.970759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.971033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.971822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:22.972126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:22.973498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e529a-c8d3-4dc6-96a3-edc772efb668 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:22.982590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 520/1034] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:06:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:22.984148 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086947d8-5a42-4a30-9433-0dd4a2de65bc None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:22.984445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086947d8-5a42-4a30-9433-0dd4a2de65bc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:22.984722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086947d8-5a42-4a30-9433-0dd4a2de65bc None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:22.985328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b90519-bb3f-4cd4-a0ef-502c4f064955 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:22.986083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 521/1035] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:22.990564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086947d8-5a42-4a30-9433-0dd4a2de65bc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.020125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.020795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.021074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.021348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.022053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] RBAC: Adding query filter params (name=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:23.022362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.024188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624071ea-9554-495a-8bfe-82ddaf85999d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.029059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 515/1036] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:23.030226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.030512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.030786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.048159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:23.049345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:23.062128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.097056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086947d8-5a42-4a30-9433-0dd4a2de65bc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.115412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 516/1037] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:23.116241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.134014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.134805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.135254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.135383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.146128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.147627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.151020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:23.165861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:23.166938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 522/1038] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:23.173163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.217257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edebb46-bdce-42f9-86dc-79879a76b205 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.253635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.254483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.254832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.255185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.255934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] RBAC: Adding query filter params (type=image) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:23.256245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.258510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.265778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 517/1039] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:23.267006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.279622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f265293-66e5-4a0e-8e8f-40807322bd2f None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.299803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.314653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.315468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.315746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.316016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.317171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.318609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.321061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 523/1040] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:23.321789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.364583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.369316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117bdab-c785-47d1-bea7-6218556e21db None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.405911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.406579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.406866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.407144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.407893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:23.408201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.409608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.421217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 518/1041] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:23.422255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.427890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5f160-4352-41c2-9fd6-ca7a3ff756cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.462134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.462775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.463054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.463353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.464074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.465432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.482935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 524/1042] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:06:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:23.483850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.534572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.545440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62cb8f-2c2d-446f-8e7c-3d145de61c2d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.597398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.597993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.598272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.598540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.599277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.600729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.606109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ff6b1-f4f7-48fd-9fd4-5d9ad047dcdb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.635324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 519/1043] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:06:23 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:23.658114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.659055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.659440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.659786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.669697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.671360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:23.687709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:23.688718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:23.689716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 525/1044] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:23.698189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.715872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:23.813876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb7681-32b6-4a75-99b6-6a6df97724e8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.849718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:23.850392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:23.850672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:23.851500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:23.852524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:23.852835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:23.854275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:23.872677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 520/1045] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:23.880155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:23.987675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045d3a-e21f-43d6-98e6-3b769b62c1e8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:23.989900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:24.025644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:24.026293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:24.026570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:24.026840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:24.096425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:24.097880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:24.198081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 526/1046] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:24.203844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:24.277820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:24.311128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5c11a-3f19-4a50-a885-8cb53f5013b8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:24.345894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:24.346678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:24.346952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:24.347217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:24.357739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:24.359187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:24.379590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:24.380511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:24.381424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:24.381424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:24.381424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:24.381424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:24.382762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 521/1047] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:24.387336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:24.390892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:24.487885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8a0dc-fb57-4c04-bb60-caf1667d16cf None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:24.522326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:24.523015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:24.523329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:24.523611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:24.524338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:24.524650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:24.526095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:24.542866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 527/1048] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:24.544280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:24.644589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31691-efdc-450c-9872-7663f1475682 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:24.667469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:24.681355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:24.682165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:24.682446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:24.682718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:24.690171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:24.691656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:24.705826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:24.706727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:24.707684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:24.711628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 522/1049] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:24.713363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:24.781768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:24.825662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b593e-e392-4ae8-97cc-509a0c391772 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:24.859618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:24.860278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:24.860560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:24.860837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:24.861624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:24.861931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:24.863388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb1a3-64c1-4d04-b976-35ca8b3a72de None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:24.875897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 528/1050] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:24.881874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:24.981877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1edb95-b28e-4ed2-b7cb-4e2b9d405e65 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:25.015742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:25.016392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:25.016676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:25.016949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:25.017765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] RBAC: Adding query filter params (role.id=61570e75122a4b788521b019751d7dc3, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:25.018076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:25.019522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556db1af-e4d5-4c3b-a374-d73bcc4d4780 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:25.024907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 523/1051] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:24 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=61570e75122a4b788521b019751d7dc3 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:25.785688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0ffc-328c-49b8-9385-b7bcf816e91f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:25.786205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0ffc-328c-49b8-9385-b7bcf816e91f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:25.786284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a0ffc-328c-49b8-9385-b7bcf816e91f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:25.787825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 529/1052] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:25.792994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc0a6-70ec-48c3-af98-0f31ef2029db None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:25.793392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc0a6-70ec-48c3-af98-0f31ef2029db None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:25.793558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc0a6-70ec-48c3-af98-0f31ef2029db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:25.810825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f70dc0a6-70ec-48c3-af98-0f31ef2029db None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:25.812032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f70dc0a6-70ec-48c3-af98-0f31ef2029db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:25.854990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:25.880097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 524/1053] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:25.902394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0ffc-328c-49b8-9385-b7bcf816e91f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:25.911310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f34cc8-b819-49f2-8f40-772cc1f0de45 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:25.911611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f34cc8-b819-49f2-8f40-772cc1f0de45 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:25.911893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f34cc8-b819-49f2-8f40-772cc1f0de45 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:25.914006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 525/1054] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:25.919131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d112a-d266-4119-841e-72972faa5a3b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:25.919477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d112a-d266-4119-841e-72972faa5a3b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:25.919756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d112a-d266-4119-841e-72972faa5a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:25.937047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-526d112a-d266-4119-841e-72972faa5a3b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:25.938236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-526d112a-d266-4119-841e-72972faa5a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:26.008290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 526/1055] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:26.019397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a0ffc-328c-49b8-9385-b7bcf816e91f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.031549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d112a-d266-4119-841e-72972faa5a3b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.037247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.049557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.050210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.050490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.050766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.127679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.129194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.145963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d112a-d266-4119-841e-72972faa5a3b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.154870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.185125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.186057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.186363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.186607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.197816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:26.205358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:26.206293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:26.208373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 527/1056] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:26.212658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.243447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 530/1057] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:26.245492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.270926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.324429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c6c2-eacf-494d-a49f-f3724b50bae3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.354801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17e902-c1a4-4b60-a958-55adc47f63c4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.357537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.358192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.358469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.358740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.359505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:26.359810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.361174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.376851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 528/1058] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:26.377916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.399523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.400317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.400847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.400847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.407295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.411793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.413841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:26.431780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:26.432708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:26.433595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:26.433595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:26.433595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:26.433595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:26.433595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 531/1059] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:26.439099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.489696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2886d-753b-4bc6-bc21-689a472d4f26 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.518035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.530479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.531387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.531668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.531940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] PATH_INFO: `/v3/users/1c0f07cf394441f082702e0aecb2b743` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.542001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bfb-8286-425d-90f4-3685ebda4aea None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.546794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] RBAC: Authorizing `identity:get_user(user_id=1c0f07cf394441f082702e0aecb2b743)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.548281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.550139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 529/1060] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/users/1c0f07cf394441f082702e0aecb2b743 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:26.556234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:26.584343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.584935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.585248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.585528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.586255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:26.586567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.588020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.602026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 532/1061] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:26.655805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.674013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e349-0d32-4141-b5e8-4e263b27573c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:26.712915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:26.713587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:26.713873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:26.714151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:26.787770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:26.789758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:26.878890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:26.895456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 530/1062] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 03:06:26.901539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.004281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81ff6e-4ded-4615-b102-5f50992ba9b5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:27.037935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:27.038877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.039157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.039465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.040730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:27.042207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792af155-afa1-4f95-9d98-f7c1fd52eaa3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:27.044053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 533/1063] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:26 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:27.050846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.150504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d9558-d4fc-4c35-8fba-a9c08d4de0bb None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:27.183340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:27.183985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.184268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.184546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.185360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=4812148c2eec4432816d6e4f19aa5c3d, user.id=1c0f07cf394441f082702e0aecb2b743) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:27.185670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:27.187098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637d8bb-ba74-4138-aa22-8277ab141275 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:27.192425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 531/1064] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:27 2022] GET /identity/v3/role_assignments?user.id=1c0f07cf394441f082702e0aecb2b743&scope.project.id=4812148c2eec4432816d6e4f19aa5c3d&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:27.241759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71bb8b-e4ee-4173-8fb5-982c72e8cbaf None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.242106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71bb8b-e4ee-4173-8fb5-982c72e8cbaf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.242517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71bb8b-e4ee-4173-8fb5-982c72e8cbaf None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.244429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 534/1065] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:27.252141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.252430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.252709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.271612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:27.273363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:27.323595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:27.352653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 532/1066] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:27.375687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71bb8b-e4ee-4173-8fb5-982c72e8cbaf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.380874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:27.481166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71bb8b-e4ee-4173-8fb5-982c72e8cbaf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:27.513803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:27.514604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.514879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.515437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.525672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:27.533674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:27.534626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:27.535509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:27.539418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 535/1067] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:06:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:27.541697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.639729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c174618-0bc1-4c52-8f20-78d1c63f1076 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:27.678246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:27.679536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.679817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.680192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.680933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:27.681234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:27.682593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:27.690418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 533/1068] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:06:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:27.696614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.757225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:27.794672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb5c17-4c61-4984-bd26-d04572f6f8d3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:27.827046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:27.827864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:27.828151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:27.828422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:27.839329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:27.840778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:27.857401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:27.858458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:27.859404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:27.859404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:27.859404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:27.859404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:27.859404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 536/1069] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:27.865010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:27.908736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:27.971264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb31ece-d873-4de0-b93b-14cc7985d91b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.005280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.005940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.006223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.006498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.007940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:28.008285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.009655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.021736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 534/1070] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:28.027795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.127678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64527b25-e038-4182-8db5-5cd9ea7c2ad2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.143910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:28.161410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.162183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.162460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.162728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.170203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.171667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:28.185694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:28.186817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:28.187826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 537/1071] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:28.192630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.291440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5733da-cda1-43c5-8eef-c666ffa687a4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.293062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:28.332430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.333093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.333373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.333651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.334482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:28.334789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.336267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.353541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 535/1072] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:28.359233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.458476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695952-e327-4dd7-859a-7b8b11631af8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.490895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.492332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.492611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.492881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.503047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.504524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:28.521308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:28.522191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:28.523256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:28.523256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:28.523256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:28.523256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:28.523256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 538/1073] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:28.528340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.626085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbaa38-d730-4936-a66e-dcdcde999375 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.659074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.659749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.660032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.660307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.661019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:28.661323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.662689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.674194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 536/1074] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:28.680079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.707985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:28.782372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22363c-292b-40ce-853c-40cc2f19c9a1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.815782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.816568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.816847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.817117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.823921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.825366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:28.839861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:28.842347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:28.843973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:28.847580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 539/1075] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:28.850278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:28.873230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:28.959799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294a40-edf3-4a70-b49d-d3369fe561d0 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:28.993779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:28.994434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:28.994711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:28.994978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:28.995790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:28.996098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:28.997526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e26f88-6c02-4e11-a8e4-77f976604c76 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:29.010810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 537/1076] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:29.016831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:29.103843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:29.116760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aadaba-b8aa-421b-aeb7-9823f697e353 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:29.150718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:29.151625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.151913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.152186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/61570e75122a4b788521b019751d7dc3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.154341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=2e9e7e21bd294ab389a68c50b4c2131a, role_id=61570e75122a4b788521b019751d7dc3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:29.155804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a191dcf-ef3f-4479-8f42-534245a236be None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:29.168166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 540/1077] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:06:29 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/2e9e7e21bd294ab389a68c50b4c2131a/roles/61570e75122a4b788521b019751d7dc3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:06:29.405810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34701e7-8f1b-450d-a1b8-60d9b7df7f67 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.406097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34701e7-8f1b-450d-a1b8-60d9b7df7f67 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.406378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34701e7-8f1b-450d-a1b8-60d9b7df7f67 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.407768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 538/1078] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:29.413196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.413615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.413894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.430791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:29.431998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:29.498316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 541/1079] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:29.520715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34701e7-8f1b-450d-a1b8-60d9b7df7f67 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:29.602277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:29.624227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34701e7-8f1b-450d-a1b8-60d9b7df7f67 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:29.658294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:29.659082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.659394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.659666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.669702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:29.677230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:29.678273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:29.679763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 539/1080] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:29.685013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:29.760478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:29.782654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5667762-29c5-4c84-b7d9-dbf946942d63 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:29.815168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:29.815853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.816137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.816413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.817169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:29.817476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:29.818861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:29.827843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 542/1081] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:29.833622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:29.932501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111443-0a3c-4120-a241-bbc885207714 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:29.968961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:29.969827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:29.970109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:29.970387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] PATH_INFO: `/v3/users/1c0f07cf394441f082702e0aecb2b743` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:29.971594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] RBAC: Authorizing `identity:get_user(user_id=1c0f07cf394441f082702e0aecb2b743)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:29.973036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:29.974822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 540/1082] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:29 2022] GET /identity/v3/users/1c0f07cf394441f082702e0aecb2b743 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:29.980979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:30.078369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69ba61-bef1-4783-bdcc-00205d68fdfa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:30.111361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:30.112245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:30.112538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:30.112833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:30.114144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:30.115645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d34a6-0f91-420f-97ef-2266e7b08882 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:30.117634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 543/1083] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:29 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:30.123562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:30.134841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:30.221261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b73f6-f30c-4915-83eb-40a5f113971b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:30.253973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:30.254875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:30.255161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:30.255465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d/users/1c0f07cf394441f082702e0aecb2b743/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:30.257661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] RBAC: Authorizing `identity:create_grant(project_id=4812148c2eec4432816d6e4f19aa5c3d, user_id=1c0f07cf394441f082702e0aecb2b743, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:30.259144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4e923-e852-4dec-9795-2fdc09ae7723 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:30.274895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 541/1084] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:06:30 2022] PUT /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d/users/1c0f07cf394441f082702e0aecb2b743/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:06:31.362809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f4ae-13fe-454d-8341-aab58e2c2aad None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:31.363443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f4ae-13fe-454d-8341-aab58e2c2aad None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:31.363533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f4ae-13fe-454d-8341-aab58e2c2aad None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:31.366846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 544/1085] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:31.371524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:31.372392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:31.372673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:31.390897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:31.392136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:31.453156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:31.459000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 542/1086] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:31.484123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f4ae-13fe-454d-8341-aab58e2c2aad None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:31.495763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:31.585569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f4ae-13fe-454d-8341-aab58e2c2aad None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:31.618297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:31.619035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:31.619347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:31.619624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:31.629519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:31.637711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:31.638781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:31.644862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 545/1087] 10.222.0.39 () {56 vars in 1120 bytes} [Thu May 26 03:06:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:31.647094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:31.744882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92bd8b-e500-4566-a3b5-52d40a281169 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:31.778787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:31.779482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:31.779762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:31.780033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:31.780769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:31.781072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:31.782423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:31.790828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 543/1088] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:06:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:31.796754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:31.862466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:31.895592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ff58e-5db4-4e1f-9385-85b250d29b0d None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:31.930326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:31.931118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:31.931438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:31.931714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:31.942302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:31.943793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:31.961643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:31.962595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:31.963946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:31.963946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:31.963946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:31.963946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:31.963946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 546/1089] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:31.968428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.028382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.067899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad825e-c2da-45c3-9037-4965b76d1f5a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.101279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.101935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.102215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.102489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.103199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:32.103532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.104889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.117068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 544/1090] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:32.122831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.221359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21f259-fc2c-4602-9216-48d1e6d590f5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.239986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.254035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.254822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.255101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.255407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.263393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.264832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:06:32.279057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:32.280301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:32.286678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 547/1091] 10.222.0.39 () {56 vars in 1093 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:32.287440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.390637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccba82-ec68-403e-a144-dbd9bdea1390 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.411068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.425166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.425841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.426126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.426405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.427257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:32.427580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.429576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dd300-abfa-4ab3-b09d-8f77df593d0d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.447262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 545/1092] 10.222.0.39 () {56 vars in 1133 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:32.452950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.473924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01c750-cd7b-4d2b-b7cf-b57061243427 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.474145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01c750-cd7b-4d2b-b7cf-b57061243427 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.474425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01c750-cd7b-4d2b-b7cf-b57061243427 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.475867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 546/1093] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:06:32.480927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.481216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.481494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.498185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:32.499412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:32.554452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ab890-ae9c-4a91-9ddb-aa45a321cf45 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.564183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 547/1094] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:32.587854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.588051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.588500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.588783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.589060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.635800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.655706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.657173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.684893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecfdfd-f78c-482f-b7bb-e8523bff2b7c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.693918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.720453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.721241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.721527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.721804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.732021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:32.739535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:32.740542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:32.741442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 548/1095] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:32.746625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.758651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 548/1096] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:32.764259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.841044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.845832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a74d0-766a-4d44-ad98-8895b866a12e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.861125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be21071-b683-4a7e-b3b6-6ae4f42fb9ee None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:32.878753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.879445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.879730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.880004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.880740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:32.881045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.882967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.891147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 549/1097] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:32.895062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:32.895889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:32.896145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:32.896415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:32.896805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.906428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:32.907909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:06:32.925154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:32.926071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:32.926968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:06:32.926968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:06:32.926968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:06:32.926968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:32.926968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 549/1098] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:32.927287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:32.932390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:32.997724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da922ca-5978-48ed-ad4d-73acca7bec71 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.030492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cf399-a372-41e9-8dc4-71df78f3d536 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.030658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.031633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.031918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.032186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] PATH_INFO: `/v3/users/1c0f07cf394441f082702e0aecb2b743` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.033357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] RBAC: Authorizing `identity:get_user(user_id=1c0f07cf394441f082702e0aecb2b743)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.034779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.036417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 550/1099] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/users/1c0f07cf394441f082702e0aecb2b743 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:33.042813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.059689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.065161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.065815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.066095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.066372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.067091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:33.067437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.068802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.080010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 550/1100] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:33.086056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.140588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.145870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3972e-21cb-4530-a28c-4d5614978687 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.179711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.180362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.180648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.180925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.184874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f61c8-005d-4a11-bddc-fc9a340a17a2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.217653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.218429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.218707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.218978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.225748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.227221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:06:33.241679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:33.243195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:33.244132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 551/1101] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:06:33.248208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.248758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.249590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.304121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.347419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.352733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51b236-7be0-408a-be3e-84e4d73735d9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.355684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 551/1102] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 03:06:33.358120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.388148 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.388800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.389080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.389350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.390381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:33.390684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.392134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.403476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 552/1103] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:33.409379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.461723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcc466-c5c8-4e34-b6ff-25cd1c9fdbef None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.495744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.496592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.496880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.497156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.498377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.499918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.501604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 552/1104] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:33.506650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef98a0-5fcd-49cb-85af-914ab2c0a036 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.507664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:33.518948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.540608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.541262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.541554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.541831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.542655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] RBAC: Adding query filter params (role.id=61570e75122a4b788521b019751d7dc3, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=2e9e7e21bd294ab389a68c50b4c2131a) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:33.542967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.544443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e515e2-83c8-499f-82e9-9c93439babc9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.550596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 553/1105] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/role_assignments?user.id=2e9e7e21bd294ab389a68c50b4c2131a&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=61570e75122a4b788521b019751d7dc3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:33.576989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:33.613283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37489aa4-4401-4013-bb17-1ba4f267f2ee None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:33.651434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:33.652111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:33.652394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:33.652670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:33.653510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=4812148c2eec4432816d6e4f19aa5c3d, user.id=1c0f07cf394441f082702e0aecb2b743) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:33.653825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:33.655298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed83174-0515-47e0-88d8-7cd6409aa579 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:33.661863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 553/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:33 2022] GET /identity/v3/role_assignments?user.id=1c0f07cf394441f082702e0aecb2b743&scope.project.id=4812148c2eec4432816d6e4f19aa5c3d&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:35.902639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321867-c6c3-4771-b228-f6238de48c1b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:35.903161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321867-c6c3-4771-b228-f6238de48c1b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:35.903273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321867-c6c3-4771-b228-f6238de48c1b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:35.904599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 554/1107] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:35.909787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:35.910202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:35.910339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:35.927928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:35.929825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:35.993316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 554/1108] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:06:36.017653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321867-c6c3-4771-b228-f6238de48c1b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:36.117130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321867-c6c3-4771-b228-f6238de48c1b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:36.150848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:36.151643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:36.151922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:36.152193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:36.153378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:36.154742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ebf64-93aa-4933-a95f-3d455c658f9f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:36.156370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 555/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:36.162110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:36.219396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:36.259133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd679c3-cae2-48fc-bf59-f230c63bfcdc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:36.292131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:36.292782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:36.293064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:36.293331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:36.294018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:36.295457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933d5c0-639d-44c9-bd64-a302464325e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:36.314432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 555/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:06:36 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:38.556116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b3daa-71a7-4efc-ad92-fabfecbada18 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:38.556639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b3daa-71a7-4efc-ad92-fabfecbada18 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:38.556639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b3daa-71a7-4efc-ad92-fabfecbada18 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:38.558022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 556/1111] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:38.563180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:38.563593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:38.563752 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:38.580658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:38.581834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:38.646011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 556/1112] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:38.668954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b3daa-71a7-4efc-ad92-fabfecbada18 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:38.714359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:38.771052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b3daa-71a7-4efc-ad92-fabfecbada18 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:38.805441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:38.806274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:38.806548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:38.806816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] PATH_INFO: `/v3/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:38.808272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] RBAC: Authorizing `identity:get_role(role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:38.809639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:38.811008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 557/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:38 2022] GET /identity/v3/roles/986616b48b464a8c82fe68f806417c4a => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:38.816806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:38.861480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:38.913948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d565be9-0ab5-4d87-b961-a6e54578929e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:38.946459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:38.947339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:38.947620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:38.947888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] PATH_INFO: `/v3/users/403e3c3a69204305ae1910611cb8c16f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:38.960479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] RBAC: Authorizing `identity:get_user(user_id=403e3c3a69204305ae1910611cb8c16f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:38.961910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:38.963555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 557/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:38 2022] GET /identity/v3/users/403e3c3a69204305ae1910611cb8c16f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:38.969194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:39.073522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbcdd-1429-4723-a47b-d4842f9be836 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:39.086762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:39.107568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:39.108211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:39.108491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:39.108768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:39.174853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:39.176259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:39.270159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 558/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 03:06:39.276616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:39.385223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9214ac-7ebe-4d1d-942f-8b169d68c65c None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:39.387507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:39.419686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:39.420535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:39.420818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:39.421092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:39.422251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:39.423727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4c7a1-625b-4910-a80c-8234c66cb036 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:39.425449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 558/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:39 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:39.431558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:39.445782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:39.529481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389845-e621-46aa-9474-784a8d899074 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:39.565360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:39.566006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:39.566285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:39.566564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:39.567418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=4812148c2eec4432816d6e4f19aa5c3d, user.id=403e3c3a69204305ae1910611cb8c16f) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:39.567733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:39.569719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b15f7-e3a8-450c-9558-a1ddb193433f None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:39.575139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 559/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:39 2022] GET /identity/v3/role_assignments?user.id=403e3c3a69204305ae1910611cb8c16f&scope.project.id=4812148c2eec4432816d6e4f19aa5c3d&role.id=986616b48b464a8c82fe68f806417c4a => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:41.727954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a1ea-8aff-4c57-898d-39e6807ad843 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:41.728478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a1ea-8aff-4c57-898d-39e6807ad843 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:41.728478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a1ea-8aff-4c57-898d-39e6807ad843 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:41.729847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 559/1118] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:41.734932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:41.735356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:41.735508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:41.752402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:41.753592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:41.817055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 560/1119] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:41.839621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a1ea-8aff-4c57-898d-39e6807ad843 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:41.938332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a1ea-8aff-4c57-898d-39e6807ad843 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:41.971022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:41.971881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:41.972160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:41.972430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] PATH_INFO: `/v3/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:41.973880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] RBAC: Authorizing `identity:get_role(role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:41.975809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:41.977145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 560/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:41 2022] GET /identity/v3/roles/986616b48b464a8c82fe68f806417c4a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:41.982794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:41.991198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:42.077488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234f963-48a1-48fb-b656-cea531fa4aa1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:42.109647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:42.110502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:42.110786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:42.111060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] PATH_INFO: `/v3/users/403e3c3a69204305ae1910611cb8c16f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:42.112251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] RBAC: Authorizing `identity:get_user(user_id=403e3c3a69204305ae1910611cb8c16f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:42.113673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:42.116235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 561/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:41 2022] GET /identity/v3/users/403e3c3a69204305ae1910611cb8c16f => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:42.121839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:42.165770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:42.221496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d5c46-0e92-459a-a3da-228bf3c38d57 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:42.254445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:42.255260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:42.255547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:42.255821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:42.257610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:42.259041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c553b-fdb4-4ce4-8408-56721ddeda5e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:42.260737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 561/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:42 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:42.266774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:42.339208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:42.363295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3232bb3-44ac-48cb-a446-ecfe32b34df0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:42.399504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:42.400383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:42.400665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:42.400941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d/users/403e3c3a69204305ae1910611cb8c16f/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:42.403096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] RBAC: Authorizing `identity:create_grant(project_id=4812148c2eec4432816d6e4f19aa5c3d, user_id=403e3c3a69204305ae1910611cb8c16f, role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:42.404576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6060c1-6e5e-4c13-a8ea-ef276d44d97c None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:42.417071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 562/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:06:42 2022] PUT /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d/users/403e3c3a69204305ae1910611cb8c16f/roles/986616b48b464a8c82fe68f806417c4a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:06:44.623438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956237-9ad6-44ba-a13c-8aa59266b9d0 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:44.623931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956237-9ad6-44ba-a13c-8aa59266b9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:44.623931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4956237-9ad6-44ba-a13c-8aa59266b9d0 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:44.625246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 562/1124] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:44.630580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:44.631115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:44.631781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:44.648934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:44.650109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:44.713448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 563/1125] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:44.737018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956237-9ad6-44ba-a13c-8aa59266b9d0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:44.834985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4956237-9ad6-44ba-a13c-8aa59266b9d0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:44.843993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:44.869224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:44.870056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:44.870339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:44.870614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] PATH_INFO: `/v3/roles/986616b48b464a8c82fe68f806417c4a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:44.872095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] RBAC: Authorizing `identity:get_role(role_id=986616b48b464a8c82fe68f806417c4a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:44.873507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:44.874811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 563/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:44 2022] GET /identity/v3/roles/986616b48b464a8c82fe68f806417c4a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:44.880456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:44.940253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:44.977558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7436d3a-c507-4c33-a82e-d3bc4d65e067 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:45.010188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:45.011036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:45.011344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:45.011616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] PATH_INFO: `/v3/users/403e3c3a69204305ae1910611cb8c16f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:45.012766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] RBAC: Authorizing `identity:get_user(user_id=403e3c3a69204305ae1910611cb8c16f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:45.014188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:45.016044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 564/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:44 2022] GET /identity/v3/users/403e3c3a69204305ae1910611cb8c16f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:45.021742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:45.118022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d822562-62da-483d-9f40-613d8ef72bf4 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:45.151114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:45.151796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:45.152067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:45.152344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:45.195847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:45.217645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:45.219068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:45.313780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 564/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:45.320386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:45.418445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866c56-1288-4f2b-91a8-1cb8a2549d58 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:45.446883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:45.451936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:45.452782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:45.453066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:45.453338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] PATH_INFO: `/v3/projects/4812148c2eec4432816d6e4f19aa5c3d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:45.454486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] RBAC: Authorizing `identity:get_project(project_id=4812148c2eec4432816d6e4f19aa5c3d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:45.455958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d2100-a720-4701-8229-1a921941a5de None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:45.457645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 565/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:45 2022] GET /identity/v3/projects/4812148c2eec4432816d6e4f19aa5c3d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:45.464358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:45.540733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:45.563454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd165-03a4-4742-b0fb-10613b942b79 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:45.595573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:45.596214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:45.596495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:45.596764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:45.597576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] RBAC: Adding query filter params (role.id=986616b48b464a8c82fe68f806417c4a, scope.project.id=4812148c2eec4432816d6e4f19aa5c3d, user.id=403e3c3a69204305ae1910611cb8c16f) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:45.597883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:45.599323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfba28-c120-4336-8891-bc4f22e2c1ba None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:45.605457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 565/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:45 2022] GET /identity/v3/role_assignments?user.id=403e3c3a69204305ae1910611cb8c16f&scope.project.id=4812148c2eec4432816d6e4f19aa5c3d&role.id=986616b48b464a8c82fe68f806417c4a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:47.844868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfc93-dc29-47db-bebe-36d14321495a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:47.845425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfc93-dc29-47db-bebe-36d14321495a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:47.845425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acfc93-dc29-47db-bebe-36d14321495a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:47.846768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 566/1131] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:47.852678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:47.853056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:47.853238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:47.870927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:47.872154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:47.936432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 566/1132] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:47.959793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acfc93-dc29-47db-bebe-36d14321495a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:48.058026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acfc93-dc29-47db-bebe-36d14321495a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:48.090468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:48.091263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:48.091543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:48.091813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:48.092964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:48.094386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1e37e-8c91-45c2-8c89-a6d61f5cc341 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:48.096058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 567/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:48.101800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:48.168126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:48.198729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefcdb3-1491-4c07-834a-a2a576f8e88d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:48.232314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:48.232956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:48.233237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:48.233512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:48.234203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:48.235669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f315a-e4bc-4334-a116-0df595a034ef None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:48.259232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 567/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:06:48 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:50.481495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390164-3bd6-4eeb-ba8b-ea4abba87020 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:50.482003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390164-3bd6-4eeb-ba8b-ea4abba87020 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:50.482090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb390164-3bd6-4eeb-ba8b-ea4abba87020 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:50.483441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 568/1135] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:50.488675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:50.489069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:50.489227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:50.506801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:50.508010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:50.571959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 568/1136] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:50.596760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb390164-3bd6-4eeb-ba8b-ea4abba87020 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:50.662709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:50.696184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb390164-3bd6-4eeb-ba8b-ea4abba87020 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:50.728422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:50.729206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:50.729489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:50.729761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:50.730924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:50.732391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad53c6-07f0-4c34-b512-f799056ffbf9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:50.734004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 569/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:50.739988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:50.791332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:50.837013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e346ff2-8256-42d3-bdf9-7aee1ce3bcf0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:50.870235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:50.870915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:50.871198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:50.871505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:50.872202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:50.873625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b235b-7a67-4404-ba60-b7fe4b2f5afc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:50.892690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 569/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:06:50 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:06:53.123923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fde45-f697-4848-90fb-a20a17bb8c65 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.124459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fde45-f697-4848-90fb-a20a17bb8c65 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.124459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fde45-f697-4848-90fb-a20a17bb8c65 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.125845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 570/1139] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:53.131287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.131707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.131848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.148358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:53.149519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:53.213280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 570/1140] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:53.236330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fde45-f697-4848-90fb-a20a17bb8c65 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:53.335436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fde45-f697-4848-90fb-a20a17bb8c65 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:53.369073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:53.369872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.370152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.370453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.374141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:53.384625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:53.392549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:53.393664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:53.394549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 571/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:53.400353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:53.438651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:53.498744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2505-49d2-4f15-a2d0-0c37b5014963 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:53.531825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:53.532488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.532772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.533049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.533791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:53.534100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:53.535496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:53.543940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 571/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:53.549714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:53.646788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69c86a-3c44-4b93-8feb-b30dc7d0c4ca None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:53.679025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:53.679889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.680166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.680437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] PATH_INFO: `/v3/users/366cd80a096b421695e9dfc9309e7dea` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.694048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] RBAC: Authorizing `identity:get_user(user_id=366cd80a096b421695e9dfc9309e7dea)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:53.695602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:53.697201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 572/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:53 2022] GET /identity/v3/users/366cd80a096b421695e9dfc9309e7dea => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:53.703026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:53.801667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:53.804943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55afe92-2823-4c3f-b33a-0f796898c17b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:53.837740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:53.838373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:53.838652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:53.838923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:53.904292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:53.905657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:53.996455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:54.000252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 572/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:06:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:06:54.006445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:54.063342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:54.107936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de162c-d8e8-4921-b6b4-cc12f9f835fa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:54.141643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:54.142488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:54.142761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:54.143027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] PATH_INFO: `/v3/projects/6bf30bd6d4274a50b37441870d164c19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:54.144195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf30bd6d4274a50b37441870d164c19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:54.145656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d62964-c22b-4f94-9dd4-9fe629d11356 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:54.147692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 573/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:54 2022] GET /identity/v3/projects/6bf30bd6d4274a50b37441870d164c19 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:54.153763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:54.249031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde45d6-041b-4fda-b8ea-69db3ed480d9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:54.282540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:54.283173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:54.283477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:54.283747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:54.284584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=6bf30bd6d4274a50b37441870d164c19, user.id=366cd80a096b421695e9dfc9309e7dea) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:54.284890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:54.286305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a5067-eb57-4e42-b9bc-5b7c05dfcd63 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:54.291852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 573/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:06:54 2022] GET /identity/v3/role_assignments?user.id=366cd80a096b421695e9dfc9309e7dea&scope.project.id=6bf30bd6d4274a50b37441870d164c19&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:56.501745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75cc7f-71e8-4a67-8f77-9cd37a2d2c89 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:56.502289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75cc7f-71e8-4a67-8f77-9cd37a2d2c89 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:56.502289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75cc7f-71e8-4a67-8f77-9cd37a2d2c89 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:56.503710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 574/1147] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:56.509160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:56.509537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:56.509711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:56.526915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:56.528134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:56.584196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:56.593093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 574/1148] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:56.615733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75cc7f-71e8-4a67-8f77-9cd37a2d2c89 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:56.714312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75cc7f-71e8-4a67-8f77-9cd37a2d2c89 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:56.736284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:56.747650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:56.748432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:56.748712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:56.748984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:56.759305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:56.767009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:56.768293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:06:56.769392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 575/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:56.775080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:56.870667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f730b-607b-445b-932b-006244c673b2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:56.903108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:56.903778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:56.904059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:56.904331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:56.905067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:56.905372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:56.906750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:56.915031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 575/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:56.920703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:57.017171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b425-5e40-487f-a304-d3b392862460 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:57.054336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:57.055194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:57.055503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:57.055776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] PATH_INFO: `/v3/users/366cd80a096b421695e9dfc9309e7dea` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:57.056945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] RBAC: Authorizing `identity:get_user(user_id=366cd80a096b421695e9dfc9309e7dea)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:57.058381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:57.060044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 576/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:56 2022] GET /identity/v3/users/366cd80a096b421695e9dfc9309e7dea => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:57.065854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:57.119968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:57.163165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8124d-7fd3-48ce-91c0-601b35bacf0b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:57.195075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:57.195933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:57.196211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:57.196481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] PATH_INFO: `/v3/projects/6bf30bd6d4274a50b37441870d164c19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:57.197617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] RBAC: Authorizing `identity:get_project(project_id=6bf30bd6d4274a50b37441870d164c19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:57.199039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b18548-7a4f-4fc0-8ad4-054db4eb2a4a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:57.200701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 576/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:06:57 2022] GET /identity/v3/projects/6bf30bd6d4274a50b37441870d164c19 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:06:57.206527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:57.255557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:57.305205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008f782-e87f-46dc-9976-553a8b7168b3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:57.337434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:57.338304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:57.338584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:57.338856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] PATH_INFO: `/v3/projects/6bf30bd6d4274a50b37441870d164c19/users/366cd80a096b421695e9dfc9309e7dea/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:57.340821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bf30bd6d4274a50b37441870d164c19, user_id=366cd80a096b421695e9dfc9309e7dea, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:57.342246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d6b1a-1484-4117-b947-489c0cc68de3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:57.357919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 577/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:06:57 2022] PUT /identity/v3/projects/6bf30bd6d4274a50b37441870d164c19/users/366cd80a096b421695e9dfc9309e7dea/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:06:59.547275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3786-d0e0-4b74-ad42-56ac6193640e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:59.547794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3786-d0e0-4b74-ad42-56ac6193640e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:59.547884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf3786-d0e0-4b74-ad42-56ac6193640e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:59.549180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 577/1154] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:06:59.554859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:59.555316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:59.555436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:59.573772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:06:59.574967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:06:59.639475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 578/1155] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:06:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:06:59.664037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3786-d0e0-4b74-ad42-56ac6193640e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:59.763713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf3786-d0e0-4b74-ad42-56ac6193640e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:59.786907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:59.797507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:59.798281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:59.798564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:59.798836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:59.808691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:06:59.816184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:06:59.817331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:06:59.819365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 578/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:06:59.823796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:06:59.863390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:06:59.920149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08934-466c-4c18-93b9-aaecb3d48c23 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:06:59.952641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:06:59.953288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:06:59.953563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:06:59.953832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:06:59.954565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:06:59.954864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:06:59.956260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:06:59.964114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 579/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:06:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:06:59.969787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:00.068997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fadbc5-eaac-47a0-8102-5df0f45cb55b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:00.101759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:00.102610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:00.102891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:00.103164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] PATH_INFO: `/v3/users/366cd80a096b421695e9dfc9309e7dea` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:00.104358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] RBAC: Authorizing `identity:get_user(user_id=366cd80a096b421695e9dfc9309e7dea)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:00.105808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:00.107444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 579/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:06:59 2022] GET /identity/v3/users/366cd80a096b421695e9dfc9309e7dea => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:00.113415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:00.209635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bbe6a-978e-49a4-b02a-edfda0a12cfd None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:00.211049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:00.242710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:00.243358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:00.243638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:00.243908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:00.307798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:00.309229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:00.405652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 580/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:07:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:07:00.412135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:00.464226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:00.509215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054891-9c81-4375-8783-129f47c270e9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:00.541469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:00.542310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:00.542588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:00.542855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] PATH_INFO: `/v3/projects/6bf30bd6d4274a50b37441870d164c19` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:00.544034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] RBAC: Authorizing `identity:get_project(project_id=6bf30bd6d4274a50b37441870d164c19)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:00.545459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc48613-6471-444f-a736-83d0a0ca1611 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:00.547112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 580/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:07:00 2022] GET /identity/v3/projects/6bf30bd6d4274a50b37441870d164c19 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:00.553249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:00.555406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:00.651456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1c79-a503-4cc7-8b83-e447ffca7cd9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:00.683945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:00.684588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:00.684868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:00.685141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:00.685962 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=6bf30bd6d4274a50b37441870d164c19, user.id=366cd80a096b421695e9dfc9309e7dea) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:00.686271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:00.687728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb15029-c7e0-4c56-87af-77bfecc76eeb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:00.694004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 581/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:07:00 2022] GET /identity/v3/role_assignments?user.id=366cd80a096b421695e9dfc9309e7dea&scope.project.id=6bf30bd6d4274a50b37441870d164c19&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:02.881828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7376-3d5e-46ab-b857-28db598823fe None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:02.882349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7376-3d5e-46ab-b857-28db598823fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:02.882464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7376-3d5e-46ab-b857-28db598823fe None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:02.883771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 581/1162] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:02.889213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:02.889855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:02.890023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:02.906838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:02.908045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:02.964549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:02.972144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 582/1163] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:02.994198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc7376-3d5e-46ab-b857-28db598823fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:03.093497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc7376-3d5e-46ab-b857-28db598823fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:03.112436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:03.126697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:03.127358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:03.127668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:03.127939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:03.192264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:03.194283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:03.290213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 582/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:07:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:07:03.296241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:03.392296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2aeb9e-8ea0-4383-9fd3-aada24113991 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:03.425395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:03.426178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:03.426455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:03.426722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:03.433445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:03.434878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f09feace-408e-4b80-be96-c48fd1666bdd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:07:03.449077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:07:03.450279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:07:03.451341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 583/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:07:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:07:03.456201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:03.477063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:03.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b96f0-434a-46f9-a6bf-dc2f62cf12b1 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:03.584148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:03.584793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:03.585076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:03.585351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:03.586062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:03.586366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:03.587752 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170d45-ac1d-4613-b236-5a375e07c678 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:03.594667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 583/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Thu May 26 03:07:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:05.766073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6934-311e-4770-88f6-70a133b8dda8 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:05.766639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6934-311e-4770-88f6-70a133b8dda8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:05.766639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c6934-311e-4770-88f6-70a133b8dda8 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:05.768024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 584/1167] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:05.773242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5658-1717-4d06-8388-c3bccc8e895c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:05.773654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5658-1717-4d06-8388-c3bccc8e895c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:05.774382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5658-1717-4d06-8388-c3bccc8e895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:05.790599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bb2e5658-1717-4d06-8388-c3bccc8e895c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:05.791797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bb2e5658-1717-4d06-8388-c3bccc8e895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:05.855553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 584/1168] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:05.878743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c6934-311e-4770-88f6-70a133b8dda8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:05.936220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:05.979452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c6934-311e-4770-88f6-70a133b8dda8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:06.011726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:06.012363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:06.012641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:06.012910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:06.013630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:06.015272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c9bfe-1a10-49d3-be9e-37857ea4ab17 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:06.028967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 585/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Thu May 26 03:07:05 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:07:08.303474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393475c1-c67b-4de2-9e68-1d8276c8bd6f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:08.304022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393475c1-c67b-4de2-9e68-1d8276c8bd6f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:08.304022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393475c1-c67b-4de2-9e68-1d8276c8bd6f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:08.305372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 585/1170] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:08.310616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:08.311029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:08.311159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:08.327754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:08.328928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:08.394884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 586/1171] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:08.418782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393475c1-c67b-4de2-9e68-1d8276c8bd6f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:08.434167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:08.519053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393475c1-c67b-4de2-9e68-1d8276c8bd6f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:08.551680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:08.552565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:08.552837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:08.553106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] PATH_INFO: `/v3/domains/1ff432184bcf4981985f5097e5221c0a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:08.563129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1ff432184bcf4981985f5097e5221c0a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:08.564521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0495ce-3db8-40b8-891c-ab399463ed10 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:08.566108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 586/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Thu May 26 03:07:08 2022] GET /identity/v3/domains/1ff432184bcf4981985f5097e5221c0a => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:08.572005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:08.617083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:08.668909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35064db-4fc6-4f21-a8a2-87b413d42050 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:08.701653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:08.702291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:08.702571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:08.702845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:08.703582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:08.705013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344aa27-2e3c-4025-bd59-dcdc2421abb2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:08.732035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 587/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:07:08 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:07:10.985165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1da93-49bf-48f4-990b-03912fdd49ed None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:10.985698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1da93-49bf-48f4-990b-03912fdd49ed None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:10.985698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1da93-49bf-48f4-990b-03912fdd49ed None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:10.987061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 587/1174] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:10.992588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:10.993020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:10.993122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:11.010404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:11.011609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:11.076765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 588/1175] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:11.101738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1da93-49bf-48f4-990b-03912fdd49ed None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:11.175806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:11.202114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1da93-49bf-48f4-990b-03912fdd49ed None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:11.234938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:11.235863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:11.236097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:11.236370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] PATH_INFO: `/v3/domains/1ff432184bcf4981985f5097e5221c0a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:11.237525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=1ff432184bcf4981985f5097e5221c0a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:11.238949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd175ec-42d3-4f27-a714-540860cd9f7a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:11.241272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 588/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Thu May 26 03:07:11 2022] GET /identity/v3/domains/1ff432184bcf4981985f5097e5221c0a => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:11.247009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:11.273842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:11.344713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda9ef3-d4db-4e1e-9cbc-b9dbfb7787b4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:11.378305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:11.378975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:11.379285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:11.379567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:11.380320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:11.381769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a172-3f75-45d9-b4d8-42d3de89bdb1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:11.404009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 589/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:07:11 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:07:13.634169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c9cf-f638-441f-b33e-c2c9b506e132 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:13.634681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c9cf-f638-441f-b33e-c2c9b506e132 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:13.634798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c9cf-f638-441f-b33e-c2c9b506e132 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:13.636121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 589/1178] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:07:13.641557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:13.641943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:13.642106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:13.659758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:13.661584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:13.735487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 590/1179] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:13.759083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c9cf-f638-441f-b33e-c2c9b506e132 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:13.856702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c9cf-f638-441f-b33e-c2c9b506e132 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:13.889264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:13.890230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:13.890507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:13.890775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:13.900987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:07:13.908358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:07:13.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:07:13.910732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 590/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:07:13.916139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:13.930705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:14.018417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ffe69-c304-491f-9f97-b183cce72453 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:14.057356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:14.058021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:14.058302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:14.058577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:14.059350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:14.059658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:14.061033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:14.068920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 591/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:14.074806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:14.077641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:14.178865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c386d-b619-45fe-9dca-f07a68e2183d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:14.210942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:14.211813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:14.212086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:14.212352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] PATH_INFO: `/v3/users/1fe77d814f4e42b697c97e4db6409162` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:14.225700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] RBAC: Authorizing `identity:get_user(user_id=1fe77d814f4e42b697c97e4db6409162)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:14.227148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:14.228781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 591/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:07:14 2022] GET /identity/v3/users/1fe77d814f4e42b697c97e4db6409162 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:14.234507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:14.308580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:14.331246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d64ca-f309-4d93-acbd-8676de2fdbeb None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:14.365499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:14.366139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:14.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:14.366692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:14.435409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:14.436769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:14.508895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:14.532218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 592/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:07:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:07:14.538746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:14.596007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:14.636022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeab31-7d76-428b-8570-bd50ca01dd8d None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:14.669269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:14.670109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:14.670392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:14.670666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] PATH_INFO: `/v3/projects/584eb43663e443ca8a4f9a972128d5a9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:14.671844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] RBAC: Authorizing `identity:get_project(project_id=584eb43663e443ca8a4f9a972128d5a9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:14.673276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df2249-1ed2-463b-8858-2ae46e90df3b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:14.674945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 592/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:07:14 2022] GET /identity/v3/projects/584eb43663e443ca8a4f9a972128d5a9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:14.681496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:14.778640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de90db-184a-48b6-8a07-86edc80c6386 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:14.812158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:14.812804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:14.813083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:14.813356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:14.814179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=584eb43663e443ca8a4f9a972128d5a9, user.id=1fe77d814f4e42b697c97e4db6409162) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:14.814491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:14.815957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8e657-2431-4d11-a643-d4783dc6cc73 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:14.821809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 593/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:07:14 2022] GET /identity/v3/role_assignments?user.id=1fe77d814f4e42b697c97e4db6409162&scope.project.id=584eb43663e443ca8a4f9a972128d5a9&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:16.970198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8a375-6f81-4ca0-888f-2510cc72159a None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:16.970722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8a375-6f81-4ca0-888f-2510cc72159a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:16.970722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8a375-6f81-4ca0-888f-2510cc72159a None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:16.972125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 593/1186] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:16.978030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:16.978455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:16.978599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:16.996075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:16.997260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:17.029799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:17.062463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 594/1187] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:17.084933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8a375-6f81-4ca0-888f-2510cc72159a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:17.183818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8a375-6f81-4ca0-888f-2510cc72159a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:17.208884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:17.216562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:17.217335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:17.217615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:17.217887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:17.228299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:07:17.235667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:07:17.236779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:07:17.237812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 594/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:07:17.243606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:17.339157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c203c6-f57d-46ed-bcfa-f6568a15d7e9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:17.372157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:17.372832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:17.373114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:17.373387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:17.374133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:17.374439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:17.375850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:17.384047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 595/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:17.390239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:17.496468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccfc58-f796-4a9c-b5f3-502ac897c53a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:17.529527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:17.530382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:17.530657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:17.530926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] PATH_INFO: `/v3/users/1fe77d814f4e42b697c97e4db6409162` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:17.532107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] RBAC: Authorizing `identity:get_user(user_id=1fe77d814f4e42b697c97e4db6409162)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:17.533538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:17.535129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 595/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:07:17 2022] GET /identity/v3/users/1fe77d814f4e42b697c97e4db6409162 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:17.541699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:17.572051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:17.638118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d36c6-551a-4460-9461-92d266e8ced0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:17.670933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:17.671806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:17.672085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:17.672355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] PATH_INFO: `/v3/projects/584eb43663e443ca8a4f9a972128d5a9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:17.673523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] RBAC: Authorizing `identity:get_project(project_id=584eb43663e443ca8a4f9a972128d5a9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:17.674946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153839c4-a8bc-47d5-b16d-64f0ee5d4f68 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:17.676653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 596/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:07:17 2022] GET /identity/v3/projects/584eb43663e443ca8a4f9a972128d5a9 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:17.682664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:17.737095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:17.780446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ad27c-0b17-4ba7-8253-bac88d786a3e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:17.813675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:17.814553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:17.814834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:17.815109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] PATH_INFO: `/v3/projects/584eb43663e443ca8a4f9a972128d5a9/users/1fe77d814f4e42b697c97e4db6409162/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:17.817956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] RBAC: Authorizing `identity:create_grant(project_id=584eb43663e443ca8a4f9a972128d5a9, user_id=1fe77d814f4e42b697c97e4db6409162, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:17.819424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc28c8-eaba-491f-8b06-b7847b1be3aa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:17.832038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 596/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:07:17 2022] PUT /identity/v3/projects/584eb43663e443ca8a4f9a972128d5a9/users/1fe77d814f4e42b697c97e4db6409162/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:07:19.994631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1859-bbcb-46d6-ba4c-c3a4f59c2670 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:19.995188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1859-bbcb-46d6-ba4c-c3a4f59c2670 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:19.995188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d1859-bbcb-46d6-ba4c-c3a4f59c2670 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:19.996584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 597/1193] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:07:20.001737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:20.002109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:20.002291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:20.019966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:07:20.021683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:07:20.092257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 597/1194] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:07:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:07:20.116110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d1859-bbcb-46d6-ba4c-c3a4f59c2670 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:20.229359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:20.233783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d1859-bbcb-46d6-ba4c-c3a4f59c2670 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:20.268815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:20.269525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:20.269806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:20.270730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:20.282300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:07:20.290116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:07:20.291219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:07:20.292468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 598/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:07:20.299521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:20.347068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:20.403636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa21c9-550c-4513-b0a1-61d9c2935f83 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:20.438445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:20.439108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:20.439418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:20.439692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:20.440439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:20.440749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:20.442865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:20.451439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 598/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:20.457270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:20.573998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aba3e-9e97-4119-aa3c-e7c9d802c607 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:20.610461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:20.611837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:20.612126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:20.612404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] PATH_INFO: `/v3/users/1fe77d814f4e42b697c97e4db6409162` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:20.614979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] RBAC: Authorizing `identity:get_user(user_id=1fe77d814f4e42b697c97e4db6409162)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:20.617509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:20.620326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 599/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Thu May 26 03:07:20 2022] GET /identity/v3/users/1fe77d814f4e42b697c97e4db6409162 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:20.626189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:20.729011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e705579-ac78-43f7-8c97-5110959ddbf5 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:20.737963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:20.767415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:20.768104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:20.768353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:20.768628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:20.838841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:20.840242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:20.936902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 599/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:07:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:07:20.943196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:20.970946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:21.042270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57540d-ed53-4c40-b2b3-37b355d58e91 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:21.075444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:21.076298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:21.076577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:21.076850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] PATH_INFO: `/v3/projects/584eb43663e443ca8a4f9a972128d5a9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:21.078017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] RBAC: Authorizing `identity:get_project(project_id=584eb43663e443ca8a4f9a972128d5a9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:21.079484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70e252-9cdf-4535-a505-a51208a2cffa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:21.081354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 600/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Thu May 26 03:07:20 2022] GET /identity/v3/projects/584eb43663e443ca8a4f9a972128d5a9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:21.088100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:07:21.098810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:21.184474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc30650-9af4-4249-9dbd-89b948cb6d91 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:07:21.217517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:07:21.218157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:07:21.218435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:07:21.218703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:07:21.219539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=584eb43663e443ca8a4f9a972128d5a9, user.id=1fe77d814f4e42b697c97e4db6409162) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:07:21.219845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:07:21.221265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8130464-81ee-4384-96ff-1e6c7d0edf03 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:07:21.227509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 600/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:07:21 2022] GET /identity/v3/role_assignments?user.id=1fe77d814f4e42b697c97e4db6409162&scope.project.id=584eb43663e443ca8a4f9a972128d5a9&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:07:31.071140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=52 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:07:31.224940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=64 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:21.649066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52794f94-456b-45be-bc45-41e97ec7ee98 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:21.649740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52794f94-456b-45be-bc45-41e97ec7ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:21.649740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52794f94-456b-45be-bc45-41e97ec7ee98 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:21.651665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 601/1201] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:21.657305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:21.657826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:21.657966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:21.684793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:21.686229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:21.795373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 601/1202] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:08:21.825932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52794f94-456b-45be-bc45-41e97ec7ee98 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:21.966286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52794f94-456b-45be-bc45-41e97ec7ee98 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:22.015382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:22.016346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:22.016760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:22.017016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:22.030972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:22.032600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:22.057014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:22.058397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:22.059644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:22.059644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:22.059644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:22.059644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:22.059644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 602/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:08:22.061717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:22.194173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57de35-bd99-43b7-bf97-86b3ae4b5fb2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:22.228069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:22.237307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:22.238069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:22.238416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:22.238762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:22.239592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:22.239965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:22.241424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f27bcc-51cd-4ec1-837e-643bfb54259c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:22.256153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 602/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:22.262380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:22.329484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:22.400504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874d91c-0c18-47c4-856c-88bed4ff0ca1 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:22.447270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:22.448300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:22.448787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:22.449248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:22.450195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:22.452095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd1060-643b-4bb8-9b35-c5d50aeaa2b3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:22.483553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 603/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Thu May 26 03:08:22 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:08:25.188139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fedd57-7b24-4ef5-ac62-f33d801a2122 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:25.188713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fedd57-7b24-4ef5-ac62-f33d801a2122 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:25.188713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fedd57-7b24-4ef5-ac62-f33d801a2122 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:25.190624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 603/1206] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:25.197236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:25.197687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:25.197794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:25.217256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:25.218564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:25.297544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 604/1207] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:25.322588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fedd57-7b24-4ef5-ac62-f33d801a2122 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:25.452460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fedd57-7b24-4ef5-ac62-f33d801a2122 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:25.490978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:25.491870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:25.492240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:25.492618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:25.507336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:25.517091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:25.518492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:25.519615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 604/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:25.524741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:25.575589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:25.653087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699fd4c-e9f9-404f-a252-d4e957db4d3c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:25.699463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:25.700167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:25.700452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:25.700737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:25.701519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:25.701851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:25.703612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:25.715063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 605/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:08:25.721876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:25.789025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:25.850536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827d170-0443-4fdb-8190-b8a138a76c8b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:25.891292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:25.892094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:25.892952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:25.893364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:25.906168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:25.907663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:25.926271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:25.927565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:25.928648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:25.928648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:25.928648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:25.928648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:25.932341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 605/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:25.933164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:26.040684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:26.043878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00556cc9-e89b-4472-985f-d2f042001e92 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:26.090316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:26.091668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:26.091668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:26.091839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:26.092577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:26.092931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:26.094509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:26.110493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 606/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:26.116039 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:26.233065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644be70-b3ea-4898-9bbd-f3d217e8ba01 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:26.234776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:26.272724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:26.273531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:26.273816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:26.274091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:26.282798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:26.285406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:26.301579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:26.302791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:26.305218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 606/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:26.310404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:26.436602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567647-e74c-4fac-aa56-a0add9399559 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:26.477051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:26.477836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:26.478122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:26.478417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:26.479553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:26.479825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:26.481527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:26.501881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 607/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:26.508495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:26.627045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009948-aa7c-4ace-8e7f-cf731c7587a9 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:26.666529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:26.667473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:26.667892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:26.668335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:26.715541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:26.764719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:26.766499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:26.892541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 607/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:08:26.894449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:26.914681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:27.008440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac60da1-5579-4642-9fbd-ef7445186a72 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:27.062871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:27.063946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:27.064964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:27.065431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:27.077895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:27.079586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:27.100177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:27.101385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:27.102526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:27.102526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:27.102526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:27.102526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:27.104213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 608/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:08:27.110166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:27.188161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:27.225340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c640d-50c9-46d9-b3be-49265e447b0a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:27.260056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:27.260697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:27.260985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:27.261252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:27.261968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:27.262271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:27.263915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:27.276084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 608/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:27.281852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:27.379994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:27.415194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308414de-a0cd-49ec-9c39-46c7543e9b29 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:27.456116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:27.457085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:27.457461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:27.457981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:27.465247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:27.466729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:27.480915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:27.481868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:27.485046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 609/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:27.488872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:27.606317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089e00b-5cd0-45f1-b995-55cfe0ad910a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:27.637741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:27.654048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:27.654738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:27.655020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:27.655322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:27.656104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:27.656418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:27.657842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406af547-7880-45a8-93a3-3c819e55f7bc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:27.676838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 609/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:27.678402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:27.808290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe144-7ecc-470d-be43-3620538bfba8 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:27.841942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:27.850665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:27.851504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:27.851838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:27.852191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:27.853212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:27.853568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:27.855226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c35ca-74e5-4eb3-b0df-b2d17d1cab7b None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:27.862259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 610/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:08:27 2022] GET /identity/v3/role_assignments?user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:30.172570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3939-3b1f-4201-8e76-3d8dfef07c7d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.173141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3939-3b1f-4201-8e76-3d8dfef07c7d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.173141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af3939-3b1f-4201-8e76-3d8dfef07c7d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.175025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 610/1220] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:30.180432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.180887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.181028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.199094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:30.200324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:30.269523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 611/1221] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:30.292126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af3939-3b1f-4201-8e76-3d8dfef07c7d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:30.404286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af3939-3b1f-4201-8e76-3d8dfef07c7d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:30.444731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:30.445479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.445763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.446036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.456305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:30.464693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:30.465883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:30.466768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 611/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:08:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:30.472738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:30.577572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080195c7-b36b-4db5-b02e-f733a70728b0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:30.602047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:30.614182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:30.614967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.615336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.615623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.616542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] RBAC: Adding query filter params (name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:30.616870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:30.618423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:30.628172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 612/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:30.634835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:30.694166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:30.753992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c3db-c26b-490c-b723-e23a83a3a407 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:30.790028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:30.790992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.791366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.791679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.803944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:30.805409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:30.825069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:30.826183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:30.827110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:30.827110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:30.827110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:30.827110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:30.827110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 612/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:30.832398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:30.945421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a51cb-b52e-4cf1-83b4-8feacf2bf704 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:30.981181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:30.981885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:30.982182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:30.982461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:30.983229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:30.983565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:30.985117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.001156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 613/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:31.009565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:31.114006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:31.121142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb5dd5-8d8e-469b-9866-6bbe7cb78bfc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:31.173522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:31.175032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:31.175370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:31.175768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:31.183992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:31.185494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:31.201437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:31.202648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:31.204030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 613/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:31.209207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:31.261731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:31.319992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2717505-90a6-4f1c-acc1-479a28bd2529 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:31.357277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:31.357978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:31.358263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:31.358553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:31.359423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:31.359742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:31.361787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.382893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 614/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:31.389199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:31.497951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e93aa-fb08-4a44-9f7e-b5b9f7359575 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:31.535809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:31.536704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:31.537021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:31.537397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:31.550669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:31.552170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:31.569494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:31.571060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:31.572757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:31.572757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:31.572757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:31.572757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:31.572757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 614/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:31.578695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:31.689667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d4d3e-229d-4ec4-b3d1-4d2fb26dc7aa None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:31.692100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:31.730852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:31.731635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:31.731916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:31.732187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:31.732908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:31.733217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:31.734589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.746838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 615/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:31.753287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:31.765426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:31.865068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba57d-abae-402a-9ebd-cc6499900449 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:31.903803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040b11-8088-4296-a832-823ba199d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:31.905008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040b11-8088-4296-a832-823ba199d01a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:31.905291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040b11-8088-4296-a832-823ba199d01a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:31.905600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59040b11-8088-4296-a832-823ba199d01a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:31.913651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59040b11-8088-4296-a832-823ba199d01a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:31.915394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59040b11-8088-4296-a832-823ba199d01a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-59040b11-8088-4296-a832-823ba199d01a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:31.931516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:31.932702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:31.940049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 615/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:31.940620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:32.055984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d464ed9-470d-4a8b-b73b-706f7785bad2 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:32.092511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:32.093328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:32.093710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:32.094111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:32.094984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:32.096240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:32.097703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c922f-dafc-4c2a-b2e9-c2f8056d8121 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:32.111764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 616/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:32.118570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040b11-8088-4296-a832-823ba199d01a None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:32.196524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:32.234300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59040b11-8088-4296-a832-823ba199d01a None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:32.269862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:32.270761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:32.271026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:32.271328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/c33cd0ddf4fc49e0b8e9d6f0863f81dd/roles/ea8e76aab98c440ba04c19f87bf78361` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:32.286661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=c33cd0ddf4fc49e0b8e9d6f0863f81dd, role_id=ea8e76aab98c440ba04c19f87bf78361)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:32.288177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a2a3-9fad-43cf-8f5e-24e44ba2687c None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:32.300920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 616/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:08:32 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/c33cd0ddf4fc49e0b8e9d6f0863f81dd/roles/ea8e76aab98c440ba04c19f87bf78361 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:08:34.685386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896d49-6c57-4c87-b43b-881c83101275 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:34.686004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896d49-6c57-4c87-b43b-881c83101275 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:34.686108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73896d49-6c57-4c87-b43b-881c83101275 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:34.687605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 617/1233] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:34.693504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:34.693916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:34.694137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:34.712170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:34.713394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:34.790490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 617/1234] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:34.816903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73896d49-6c57-4c87-b43b-881c83101275 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:34.847822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:34.936295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73896d49-6c57-4c87-b43b-881c83101275 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:34.976911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:34.977789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:34.978072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:34.978344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:34.990269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:34.998564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:34.999799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:35.000788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 618/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Thu May 26 03:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:35.006451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:35.057335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:35.114638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892d9b9-ddef-4a5d-8ece-84473f84cd67 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:35.157015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:35.158473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:35.159183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:35.159916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:35.160717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] RBAC: Adding query filter params (name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:35.161032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:35.162515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:35.174797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 618/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:35.181522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:35.278088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:35.294907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2649161-fcf6-433a-ad34-1aadf536d287 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:35.331967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:35.332776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:35.333074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:35.333329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:35.345855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:35.347343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:35.366419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:35.367461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:35.368600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:35.368600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:35.368600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:35.368600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:35.368600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 619/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:35.373875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:35.482683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee6fe-9cee-48cc-a0f2-f5901c786056 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:35.493549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:35.521000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:35.521657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:35.521993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:35.522256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:35.523095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:35.523425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:35.524909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:35.537555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 619/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:35.543334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:35.658929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc869bc-13a8-4c1e-8248-fc87565bbe03 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:35.697933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:35.698742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:35.699029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:35.699346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:35.702349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:35.709124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:35.711005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:35.730617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:35.731931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:35.732953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 620/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:08:35.738453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:35.852986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e48d6-8b3f-488a-892e-372653b84682 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:35.888803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:35.889461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:35.889753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:35.890014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:35.890846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:35.891177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:35.892775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:35.913772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 620/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:35.921869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:36.025217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61b029-6354-45e8-b20c-737c75fdcd02 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:36.060981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:36.061688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:36.061970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:36.062239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:36.135818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:36.144035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:36.145750 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:36.254212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 621/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:08:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:08:36.260303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:36.291623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:36.382803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4176a6-cbed-4682-8bb2-ecb842adc4d8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:36.421401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:36.422205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:36.422495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:36.422777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:36.433951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:36.435431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:36.455319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:36.456485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:36.457778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:36.457778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:36.457778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:36.457778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:36.457778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 621/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:36.463837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:36.572198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:36.577689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03624-ab06-4fc9-900b-f5a04bfe05c4 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:36.616567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:36.617277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:36.617598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:36.617917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:36.618691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:36.619024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:36.620564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:36.633826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 622/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:36.640198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:36.730857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:36.754930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e584e2-31d0-4f42-aedb-c955788cfc74 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:36.791130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:36.792015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:36.792301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:36.792579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:36.802658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:36.804277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:36.819285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:36.820402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:36.821966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 622/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:36.826736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:36.935544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c245a9-96b7-484f-81a6-c94d03c2919e None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:36.973980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:36.974676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:36.974950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:36.975220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:36.976070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:36.976373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:36.977906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d32dac-4bd0-4d50-85d1-e919b85c623a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:36.991486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 623/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:36.998058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:37.124983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ee20b-b0b2-4910-9d75-d4d1be0348f7 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:37.165154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:37.165640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:37.165941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:37.166227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:37.167105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] RBAC: Adding query filter params (role.id=ea8e76aab98c440ba04c19f87bf78361, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:37.167537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:37.169124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c734b4-de54-494d-b454-3ca22f3b98a6 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:37.187269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 623/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:08:36 2022] GET /identity/v3/role_assignments?user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=ea8e76aab98c440ba04c19f87bf78361 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:08:39.656579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba12253-6c6b-444f-9153-11e93b2d585d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:39.657130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba12253-6c6b-444f-9153-11e93b2d585d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:39.657130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba12253-6c6b-444f-9153-11e93b2d585d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:39.659450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 624/1247] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:39.666975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:39.667456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:39.667590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:39.673825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:39.686778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:39.688197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:39.761721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 624/1248] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:39.788563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba12253-6c6b-444f-9153-11e93b2d585d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:39.808777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:39.904034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba12253-6c6b-444f-9153-11e93b2d585d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:39.941884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:39.942650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:39.942952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:39.943215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:39.955704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:39.964704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:39.965841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:39.971864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 625/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:08:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:39.972568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:40.080698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02119d8-f511-4323-adbf-fc31dba9a4a8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:40.114116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:40.119623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:40.120351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:40.120633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:40.120866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:40.121659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:40.121967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:40.123492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:40.139490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 625/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:08:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:40.145430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:40.241491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:40.267147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d31d90-299b-4cf2-90c4-0734faf5104b None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:40.306936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:40.307756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:40.308084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:40.308372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:40.320980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:40.322603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:40.341431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:40.342504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:40.343635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:40.343635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:40.343635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:40.343635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:40.343635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 626/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:40.348745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:40.463662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d87-82d7-4781-a2a4-d5ec1295cd6b None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:40.506523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:40.507116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:40.507426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:40.507704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:40.508419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:40.508732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:40.510104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:40.524968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 626/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:40.531220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:40.639107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93974ac5-49bf-4a64-901a-04621c553230 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:40.673230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:40.682805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:40.683838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:40.684196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:40.684477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:40.692762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:40.694216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:40.709577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:40.718565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:40.719724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 627/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:40.721891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:40.791518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:40.840726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf884c-350b-444a-9bc9-a4c70fe9e2f8 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:40.880697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:40.881697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:40.881979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:40.882254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:40.883098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:40.883449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:40.884885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:40.903172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 627/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:40.909857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:41.022664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb7c5b-ce47-4ba4-b585-3c2909b97d66 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:41.063195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:41.064333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:41.064719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:41.064999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:41.149630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:41.166469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:41.168107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:41.279797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 628/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:08:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:08:41.281161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:41.388153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255e74d-9095-4e9c-b4ca-81564f7db275 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:41.403484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:41.426969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:41.427895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:41.428234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:41.428564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:41.446875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:41.449546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:41.473803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:41.474952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:41.481059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:41.481059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:41.481059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:41.481059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:41.481059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 628/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:41.481396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:41.578293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:41.593056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad497c39-bb14-4306-9675-48c05481d0df None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:41.628717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:41.629294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:41.629570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:41.629840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:41.630554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:41.630856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:41.632279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:41.644913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 629/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:41.650451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:41.752594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef689f1-27a2-40ad-a8c9-a1c47eb79154 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:41.791532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:41.792916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:41.793438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:41.793948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:41.803463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:41.804947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:41.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:41.820508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:41.821477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 629/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:41.826782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:41.923713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc2850-2263-4f33-b8b5-246b5b7e5089 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:41.956167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:41.956805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:41.957087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:41.957362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:41.958145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:41.958453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:41.959979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735751b6-84a6-44fe-b5b7-7e71ef4a5725 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:41.972544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 630/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:41.978419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:42.003980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:42.075917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c717f-9d2a-405c-8467-8df1e037f4c2 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:42.111222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:42.111896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:42.112174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:42.112444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:42.113257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:42.113562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:42.115178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9dfdc-0074-4d6f-85d0-826c65eb437a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:42.121359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 630/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:08:41 2022] GET /identity/v3/role_assignments?user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:44.315656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08e54-a8f1-420a-b899-eed0d5ce827d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:44.316191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08e54-a8f1-420a-b899-eed0d5ce827d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:44.316191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b08e54-a8f1-420a-b899-eed0d5ce827d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:44.317565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 631/1261] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:44.322744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:44.323168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:44.323298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:44.340092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:44.341263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:44.407201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 631/1262] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:44.430118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08e54-a8f1-420a-b899-eed0d5ce827d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:44.440823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:44.531421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b08e54-a8f1-420a-b899-eed0d5ce827d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:44.565105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:44.565932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:44.566164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:44.566436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:44.576256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:44.584394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:44.585469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:44.586359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 632/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:44.591611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:44.609584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:44.689781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e721cb2-f5f8-497b-8dfa-6250a4cd494b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:44.722740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:44.723413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:44.723693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:44.723965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:44.724693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:44.724994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:44.726345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:44.734018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 632/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:44.740036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:44.805548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:44.839022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf13dc-b789-4a98-8798-bd435679b440 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:44.870785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:44.871581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:44.871866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:44.872140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:44.882911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:44.884390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:44.901855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:44.902878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:44.904017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:44.904017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:44.904017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:44.904017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:44.904017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 633/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:44.908810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:44.986572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:45.005583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4b3c-595a-46b2-9fa8-cfc259b40e68 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.038040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.038689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.038970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.039268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.039963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:45.040265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.041616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.053538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 633/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:08:45.059160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.162037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537dd05-9d15-46ae-b851-bb0aa4500728 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.184366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:45.200186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.201077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.201434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.201718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.210729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.212217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:45.226931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:45.228058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:45.229238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 634/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:45.234081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.331999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b96dcc-6d2f-4ff4-a2cd-ceb6d0268eaa None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.364966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.365641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.365921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.366193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.367027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:45.367363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.368788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.377565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:45.386074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 634/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:45.392579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.488651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5e947-ef52-423b-970d-9fcfd6d8bb13 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.520871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.521642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.521919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.522190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.533000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.534539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:45.551821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:45.552863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:45.553947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:45.553947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:45.553947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:45.553947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:45.553947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 635/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:45.558488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.655313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a2088-d2f9-40e7-8602-b984c0e1b310 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.689315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.689961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.690238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.690506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.691211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:45.691542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.692897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.703549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 635/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:45.709365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.740329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:45.811926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d939-08f4-4bb7-96e2-1010c205b2f0 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:45.844332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:45.845195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:45.845472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:45.845742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:45.852153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:45.853585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:45.867749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:45.868778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:45.869983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 636/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:45.874973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:45.932700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:45.973411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8771b0-939d-4451-9188-54ed6aafa9cf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:46.006389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:46.006971 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:46.007274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:46.007557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:46.008331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:46.008640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:46.010054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f260-11b5-479f-9891-fbe8c529f934 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:46.021904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 636/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:46.027930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:46.117259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:46.130727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515abe6-971f-4e68-b0c3-d4c2c941b9be None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:46.170405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:46.171303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:46.171860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:46.172138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] PATH_INFO: `/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/c33cd0ddf4fc49e0b8e9d6f0863f81dd/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:46.174623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c16d6b8e11b44d9a961a2be0a71b1e74, user_id=c33cd0ddf4fc49e0b8e9d6f0863f81dd, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:46.176075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d40d02-cf80-4bcb-b0d3-ee3d4d3fe6f8 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:46.187892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 637/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Thu May 26 03:08:46 2022] PUT /identity/v3/projects/c16d6b8e11b44d9a961a2be0a71b1e74/users/c33cd0ddf4fc49e0b8e9d6f0863f81dd/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:08:48.370668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a01a-8df0-4a56-9c98-35395f28e6de None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:48.371154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a01a-8df0-4a56-9c98-35395f28e6de None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:48.371269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5a01a-8df0-4a56-9c98-35395f28e6de None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:48.372618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 637/1274] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:48.378650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:48.379027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:48.379192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:48.395801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:48.396967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:48.462238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 638/1275] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:08:48.486091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5a01a-8df0-4a56-9c98-35395f28e6de None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:48.583634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5a01a-8df0-4a56-9c98-35395f28e6de None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:48.600378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:48.616810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:48.617579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:48.617855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:48.618121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:48.628327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 03:08:48.635705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:48.636719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:48.637556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 638/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Thu May 26 03:08:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:48.643313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:48.721262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:48.740026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378ba-505e-420a-9345-fb29d3ec003d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:48.772046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:48.772697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:48.772975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:48.773245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:48.773980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:48.774279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:48.775670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:48.783881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 639/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Thu May 26 03:08:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:48.789486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:48.884878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68cb1-e67c-406e-862e-a2300dc46c79 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:48.917963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:48.918722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:48.919000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:48.919295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:48.929885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:48.931329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:48.947745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:48.948769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:48.949886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:48.949886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:48.949886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:48.949886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:48.949886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 639/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:48.954950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:49.050367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d3ab3-9537-4be5-b6c4-b8e134bf96c5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:49.082557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:49.083194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:49.083503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:49.083773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:49.084478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:49.084780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:49.086124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:49.097419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 640/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:49.102858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:49.134610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:49.210496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e97ee7-8b0b-4134-8775-65abd5019207 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:49.243002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:49.243828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:49.244111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:49.244383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:49.252124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:49.253574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 03:08:49.267867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:49.268987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:49.270170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 640/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:49.274768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:49.278479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:49.372099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945e90-6ceb-42ee-81d3-0017c8c508d9 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:49.405661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:49.406318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:49.406595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:49.407413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:49.408247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:49.408549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:49.409987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:49.427003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 641/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Thu May 26 03:08:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:49.432677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:49.522228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:49.532602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec8f14-1387-43cc-af2f-bb2291916ecf None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:49.565107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:49.565730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:49.566007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:49.566275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:49.631155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:49.632610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:49.718006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:49.729110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 641/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Thu May 26 03:08:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 03:08:49.735179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:49.800250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:49.838489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ee309-178a-4300-97e4-f7045c62ea54 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:49.871663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:49.872440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:49.872721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:49.872994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:49.882625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:49.884096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 03:08:49.901548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:49.902589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:49.903581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 03:08:49.903581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 03:08:49.903581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 03:08:49.903581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:49.903581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 642/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:49.909447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:50.004328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285a5c2-729a-4328-ba84-dd192666fddc None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:50.037337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:50.037979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:50.038260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:50.038535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:50.039270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:50.039570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:50.040920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:50.051830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 642/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Thu May 26 03:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:50.057269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:50.158491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eaa5b-3fc1-4cbc-b74d-df4cd89467c5 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:50.172341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:50.192785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:50.193566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:50.193851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:50.194135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:50.200644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:50.202124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 03:08:50.215654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application with Lock( May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:50.216660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: ERROR keystone.server.flask.application May 26 03:08:50.217614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 643/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Thu May 26 03:08:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:50.223437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:50.272179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:50.321660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4baed-46cc-4745-b583-f1f79864035e None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:50.354551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:50.355199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:50.355509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:50.355784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:50.356558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:50.356866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:50.358295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16346908-3d6b-4ac5-b534-182f1bed1f0f None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:50.371462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 643/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Thu May 26 03:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:50.377805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:50.473882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928adc64-e890-47ee-991d-2802b71afcd3 None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:50.504922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:50.505543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:50.505818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:50.506087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:50.506905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] RBAC: Adding query filter params (role.id=85e1377baa9b4d50a7aa92fe0ba16e7e, scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74, user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:50.507210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:50.508660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fea68-95fb-41ea-a6f2-b05fa3995f51 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:50.514735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 644/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Thu May 26 03:08:50 2022] GET /identity/v3/role_assignments?user.id=c33cd0ddf4fc49e0b8e9d6f0863f81dd&scope.project.id=c16d6b8e11b44d9a961a2be0a71b1e74&role.id=85e1377baa9b4d50a7aa92fe0ba16e7e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:52.704092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1446e-576e-4ceb-af55-c4ce8a6700d9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:52.704619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1446e-576e-4ceb-af55-c4ce8a6700d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:52.704619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1446e-576e-4ceb-af55-c4ce8a6700d9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:52.706000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 644/1288] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:52.711076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:52.711539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:52.711657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:52.728440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:52.729613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:52.759541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:52.795610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 645/1289] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:08:52.818682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1446e-576e-4ceb-af55-c4ce8a6700d9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:52.916722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1446e-576e-4ceb-af55-c4ce8a6700d9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:52.927507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:52.950076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:52.950857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:52.951138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:52.951435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:52.952079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:52.953456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:52.964983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:52.966172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:52.967060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 645/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:08:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:08:52.973067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:53.070566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba8b7b-bc50-43f7-ae00-a163af5e8b7c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:53.104177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:53.104819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:53.105097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:53.105368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:53.106060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:53.106362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:53.107755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8139b1-bbfe-4dd1-aedf-548fbfe27f3e None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:53.112702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 646/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Thu May 26 03:08:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:53.118263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:53.228084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed4d4-9ab9-488c-b602-2294e3195613 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:53.260172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:53.260824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:53.261102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:53.261372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] PATH_INFO: `/v3/services` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:53.262064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:53.262375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:53.264012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7ece6-0be3-4721-bf55-a3b42ff110f0 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:53.268910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 646/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Thu May 26 03:08:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:55.445222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1786-3d47-4c57-a191-5a7fae9f3d60 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:55.445713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1786-3d47-4c57-a191-5a7fae9f3d60 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:55.445802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f1786-3d47-4c57-a191-5a7fae9f3d60 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:55.447145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 647/1293] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:55.452502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16c430-58c6-4f50-89cf-45f6d6ff7c24 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:55.452875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16c430-58c6-4f50-89cf-45f6d6ff7c24 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:55.453062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16c430-58c6-4f50-89cf-45f6d6ff7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:55.470088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9e16c430-58c6-4f50-89cf-45f6d6ff7c24 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:55.471302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e16c430-58c6-4f50-89cf-45f6d6ff7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:55.499385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:55.536743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 647/1294] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:08:55.559772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f1786-3d47-4c57-a191-5a7fae9f3d60 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:55.591401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:55.660459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f1786-3d47-4c57-a191-5a7fae9f3d60 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:55.693350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:55.693998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:55.694278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:55.694552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:55.695296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:55.696662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813c0e1-2dc4-474f-afe5-a3ab36a5576d None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:55.709109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 648/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Thu May 26 03:08:55 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:08:57.923258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1b55-e15b-4abc-97dd-9283dc5c046e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:57.923790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1b55-e15b-4abc-97dd-9283dc5c046e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:57.923900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1b55-e15b-4abc-97dd-9283dc5c046e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:57.925179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 648/1296] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:08:57.930357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:57.930766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:57.930903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:57.947204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:08:57.948396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:08:58.018681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 649/1297] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:08:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 03:08:58.041356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb1b55-e15b-4abc-97dd-9283dc5c046e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:58.153108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb1b55-e15b-4abc-97dd-9283dc5c046e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:58.190570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:58.191307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:58.191587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:58.191857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:58.192497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:58.193849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:08:58.205913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:08:58.207088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:08:58.208325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 649/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:08:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 03:08:58.213216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:58.230906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:58.310345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b2e46-cc56-4a48-8ae8-1d264d46c518 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:58.343594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:58.344416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:58.344699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:58.344974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:58.345677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:58.345982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:58.347374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:58.352399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 650/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Thu May 26 03:08:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:58.358279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:58.389350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:58.456459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bd47b-efb2-4151-82da-f4c098be60a3 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:58.488087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:58.488730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:58.489009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:58.489278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:58.490054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] RBAC: Adding query filter params (interface=public, service_id=05681a7def984893b31c7e554b371d02, region_id=RegionOne) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:08:58.490359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:58.491750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a91a9-8aff-4fc8-805f-56b729edcb0a None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:58.496604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 650/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Thu May 26 03:08:58 2022] GET /identity/v3/endpoints?service_id=05681a7def984893b31c7e554b371d02&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:08:58.502827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:08:58.582187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:08:58.600413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8240c-8cc9-43b8-affb-40104ba2838a None admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:08:58.633732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:08:58.634382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:08:58.634665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:08:58.634941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:08:58.635659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:08:58.637030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de1069-0707-4244-b929-d64119b8cc37 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:08:58.644142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 651/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Thu May 26 03:08:58 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:09:00.810386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5271-cb46-4d3b-9cca-4d8cdbe3a4bd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:09:00.810937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5271-cb46-4d3b-9cca-4d8cdbe3a4bd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:09:00.810937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f5271-cb46-4d3b-9cca-4d8cdbe3a4bd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:09:00.812319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 651/1302] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:09:00.817523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:09:00.817928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:09:00.818082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:09:00.835072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:09:00.836312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:09:00.903167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 652/1303] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:09:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 03:09:00.925981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f5271-cb46-4d3b-9cca-4d8cdbe3a4bd None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:09:01.019003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:09:01.025719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f5271-cb46-4d3b-9cca-4d8cdbe3a4bd None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:09:01.058144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:09:01.058916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:09:01.059195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:09:01.059498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:09:01.060136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:09:01.061542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 03:09:01.073095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 03:09:01.074252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.get_or_create( May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application with Lock( May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self._enter() May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application return self.creator() May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application created_value = creator( May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: ERROR keystone.server.flask.application May 26 03:09:01.075438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 652/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Thu May 26 03:09:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 03:09:01.080714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:09:01.159698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:09:01.181301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d8d3c-eca2-46b8-ac9a-cac341faaeb6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:09:01.214883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:09:01.215555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:09:01.215832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:09:01.216101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] PATH_INFO: `/v3/services` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:09:01.216791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:09:01.217090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:09:01.218446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbea92e-ea47-4785-8d85-4747995cda96 None admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:09:01.223643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 653/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Thu May 26 03:09:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:09:01.229360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:09:01.326498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0ac5-6f82-41a5-83e3-175c9dda8b91 None admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:09:01.359735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:09:01.360460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:09:01.360740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:09:01.361012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:09:01.361737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:09:01.363089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2d8d2-ee29-4bf6-89df-cd730a9459c7 None admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:09:01.382595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 653/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Thu May 26 03:09:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:09:11.220597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=30 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:09:11.378237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:05.175919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7d68-eefd-46b2-9c4a-f9fd97139fc4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:05.176514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7d68-eefd-46b2-9c4a-f9fd97139fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:05.176514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e7d68-eefd-46b2-9c4a-f9fd97139fc4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:05.181717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 654/1307] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:05.183139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11e56-e09d-4ccb-9c12-a3c31df33b85 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:05.183616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11e56-e09d-4ccb-9c12-a3c31df33b85 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:05.183713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c11e56-e09d-4ccb-9c12-a3c31df33b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:05.269114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-57c11e56-e09d-4ccb-9c12-a3c31df33b85 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:05.270463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-57c11e56-e09d-4ccb-9c12-a3c31df33b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:05.403451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 654/1308] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:10:05.469512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:05.469512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:05.469512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:05.478046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 655/1309] 10.222.0.39 () {54 vars in 964 bytes} [Thu May 26 03:10:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:05.484342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488c8ab-2327-4aef-8f72-c6edfbcaee42 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:05.484640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488c8ab-2327-4aef-8f72-c6edfbcaee42 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:05.484921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488c8ab-2327-4aef-8f72-c6edfbcaee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:05.578309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f488c8ab-2327-4aef-8f72-c6edfbcaee42 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:05.579353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f488c8ab-2327-4aef-8f72-c6edfbcaee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:05.602684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:05.671396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 655/1310] 10.222.0.39 () {58 vars in 1061 bytes} [Thu May 26 03:10:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:05.680915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:05.683375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:05.817557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c74216-7ff3-4671-bf4b-6c87d4e1c84d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:05.894912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:05.895691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:05.896040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:05.896381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:05.898133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:05.955687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:06.016409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:06.017985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f5b44-9721-47a5-bcf1-85195c577e02 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:06.038442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 656/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:10:08.536573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:08.537020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:08.537313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:08.538928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 656/1312] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:08.544281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361865-602d-440b-9c75-3d19e08dbc1f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:08.544734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361865-602d-440b-9c75-3d19e08dbc1f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:08.544816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361865-602d-440b-9c75-3d19e08dbc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:08.624016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c9361865-602d-440b-9c75-3d19e08dbc1f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:08.625397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c9361865-602d-440b-9c75-3d19e08dbc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:08.731282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 657/1313] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:08.767492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:08.769024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:08.877929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d7d11-ee99-442a-9447-54b1067c5597 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:08.952571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:08.955544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:08.956506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:08.957132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:08.957724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:08.959852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:09.076168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:09.078404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e515-1622-4d99-b7f9-0473d1db9faa service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:09.098543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 657/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:11.723513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:11.724076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:11.724076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:11.725443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 658/1315] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:11.730501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021853a4-034a-4ced-8981-adc5aa36be81 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:11.730923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021853a4-034a-4ced-8981-adc5aa36be81 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:11.731060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021853a4-034a-4ced-8981-adc5aa36be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:11.795091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-021853a4-034a-4ced-8981-adc5aa36be81 None None] MFA Rules not processed for user `033961e12b9349dbba61d19814615bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:11.796416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-021853a4-034a-4ced-8981-adc5aa36be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:11.849353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:11.895082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 658/1316] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:11.935311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:11.937885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:11.947497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:12.040014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0a77e-bcdf-440c-9a0e-d2f175ccc95a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:12.113382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:12.114018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:12.114296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:12.114567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:12.116237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:12.177048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:12.223007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:12.224514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4f368-06eb-4428-be04-916d8ea199af service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:12.243105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 659/1317] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:14.996780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:14.997314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:14.997314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:14.998677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 659/1318] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:15.004286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511689f1-591e-49ab-810d-de8f3070b006 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:15.004705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511689f1-591e-49ab-810d-de8f3070b006 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:15.004852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511689f1-591e-49ab-810d-de8f3070b006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:15.068002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-511689f1-591e-49ab-810d-de8f3070b006 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:15.069198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-511689f1-591e-49ab-810d-de8f3070b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:15.169273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 660/1319] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:15.207907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:15.209450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:15.313743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132d6b2-4b56-49ca-a27c-33ed6476e056 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:15.387154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:15.387826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:15.388112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:15.388392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:15.390354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:15.397726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:15.498239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:15.499888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae39e9-45c5-4d02-8f41-3e2a4756cd3d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:15.517393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 660/1320] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:18.407042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:18.407587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:18.407702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:18.409333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 661/1321] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:18.414791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be764-1a95-467d-a3b1-31aa49ad4464 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:18.415217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be764-1a95-467d-a3b1-31aa49ad4464 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:18.415379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39be764-1a95-467d-a3b1-31aa49ad4464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:18.479675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d39be764-1a95-467d-a3b1-31aa49ad4464 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:18.480858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d39be764-1a95-467d-a3b1-31aa49ad4464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:18.540225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:18.579332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 661/1322] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:18.615889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:18.617455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:18.638555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:18.727341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fba9c-b79f-4998-b37b-2d2b3f00fd1b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:18.803184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:18.803849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:18.804126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:18.804395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:18.806030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:18.900313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:18.939730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:18.941215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85adc4-4442-45b3-badc-dacd424c271e service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:18.961634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 662/1323] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:21.231068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:21.231620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:21.231734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:21.233083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 662/1324] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:21.238658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80a591-b354-42c8-89cd-4b587824da7f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:21.239067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80a591-b354-42c8-89cd-4b587824da7f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:21.239313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80a591-b354-42c8-89cd-4b587824da7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:21.309422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ee80a591-b354-42c8-89cd-4b587824da7f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:21.310830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ee80a591-b354-42c8-89cd-4b587824da7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:21.420706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 663/1325] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:10:21.457322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:21.458995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:21.570795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb744725-a1bd-4d1f-82ae-b41aaca56fe4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:21.660010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:21.660667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:21.660954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:21.661233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:21.663480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:21.682834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:21.793659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:21.795281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d14a7-eaa0-48f7-9fc5-e9fa15a911b9 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:21.815476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 663/1326] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:24.099359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fdf8-174a-41e5-9f94-b5d6a9419ba7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:24.099893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fdf8-174a-41e5-9f94-b5d6a9419ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:24.100013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fdf8-174a-41e5-9f94-b5d6a9419ba7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:24.101454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 664/1327] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:24.106730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf941-8fbd-4d30-a58f-4f53c4747c2b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:24.107184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf941-8fbd-4d30-a58f-4f53c4747c2b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:24.107444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbf941-8fbd-4d30-a58f-4f53c4747c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:24.175298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7abbf941-8fbd-4d30-a58f-4f53c4747c2b None None] MFA Rules not processed for user `033961e12b9349dbba61d19814615bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:24.176499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7abbf941-8fbd-4d30-a58f-4f53c4747c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:24.240474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:24.281693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 664/1328] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:10:24.339602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:24.340088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:24.340404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:24.341805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 665/1329] 10.222.0.39 () {54 vars in 964 bytes} [Thu May 26 03:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:24.348551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60599e-46de-4412-a243-40b8152c219b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:24.348934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60599e-46de-4412-a243-40b8152c219b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:24.349303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60599e-46de-4412-a243-40b8152c219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:24.429939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6a60599e-46de-4412-a243-40b8152c219b None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:24.431139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6a60599e-46de-4412-a243-40b8152c219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:24.506059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 665/1330] 10.222.0.39 () {58 vars in 1061 bytes} [Thu May 26 03:10:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:24.515149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:24.516745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:24.548634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:24.624496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cebe1-f250-4f9e-853b-6814465eb60e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:24.696769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:24.697408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:24.697691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:24.697961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:24.699626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:24.776349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:24.808882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:24.810335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e6a09-ccfc-4216-ba2b-1c89caf9926c service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:24.828976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 666/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:27.735457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:27.735927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:27.736045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:27.737404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 666/1332] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:27.742913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf84f-17c0-4c0f-8ee6-81b088baf5c4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:27.743443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf84f-17c0-4c0f-8ee6-81b088baf5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:27.743538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf84f-17c0-4c0f-8ee6-81b088baf5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:27.813196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-833cf84f-17c0-4c0f-8ee6-81b088baf5c4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:27.814521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-833cf84f-17c0-4c0f-8ee6-81b088baf5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:27.909427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 667/1333] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:10:27.944951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:27.946693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:28.019076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:28.053731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf8478-0230-407f-8e04-6477568834d2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:28.129575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:28.130592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:28.130974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:28.131417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:28.133401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:28.248109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:28.249581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22eec06-6356-4ce4-84df-902dd5bf83e1 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:28.260093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:28.269466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 667/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:30.600656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:30.601234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:30.601234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:30.602597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 668/1335] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:30.607686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cda83-2056-4d12-aa4e-21926bc53ac9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:30.608107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cda83-2056-4d12-aa4e-21926bc53ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:30.608224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cda83-2056-4d12-aa4e-21926bc53ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:30.673234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8a8cda83-2056-4d12-aa4e-21926bc53ac9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:30.674430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8a8cda83-2056-4d12-aa4e-21926bc53ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:30.771023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 668/1336] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:30.811568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:30.813204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:30.852403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:30.915078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecc07-d30f-4fd3-b53d-433c69b861e4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:30.985985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:30.986550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:30.986826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:30.987098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:30.988766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:31.070728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:31.096086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:31.097512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221b0ac-2bde-469a-acfc-e6587da73d64 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:31.115520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 669/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:33.640314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:33.640900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:33.642319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:33.643161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 669/1338] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:33.648749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b91e-f95f-4e73-8809-6ce36bb8dbb1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:33.649169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b91e-f95f-4e73-8809-6ce36bb8dbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:33.649303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b91e-f95f-4e73-8809-6ce36bb8dbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:33.713482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-22a8b91e-f95f-4e73-8809-6ce36bb8dbb1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:33.714698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-22a8b91e-f95f-4e73-8809-6ce36bb8dbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:33.809577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 670/1339] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:33.846648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:33.848217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:33.923643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:33.973527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f68dec-91c8-42ae-afbb-cfe48fe7abd1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:34.047433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:34.048071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:34.048350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:34.048623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:34.050248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:34.161104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:34.162528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c46af8-3fb3-4249-886a-cf173d00f2aa service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:34.181321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 670/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:38.391603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:38.392126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:38.392229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:38.393574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 671/1341] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:38.398932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338425-3196-4f43-b1e7-32338b2daeec None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:38.399396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338425-3196-4f43-b1e7-32338b2daeec None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:38.399529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338425-3196-4f43-b1e7-32338b2daeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:38.406297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:38.472961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-96338425-3196-4f43-b1e7-32338b2daeec None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:38.474696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-96338425-3196-4f43-b1e7-32338b2daeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:38.579528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 671/1342] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:38.617633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:38.619919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:38.671729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:38.716745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358f6a8-5512-4702-98de-328f0e131de0 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:38.720448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:38.721090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:38.721366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:38.721639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:38.730745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:38.732276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2418390-5d03-4e0a-ba2a-3a4d589d9045 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:38.742314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 672/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:41.585496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21afa9-4535-49b5-9423-6ec9dc62963b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:41.586015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21afa9-4535-49b5-9423-6ec9dc62963b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:41.586015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21afa9-4535-49b5-9423-6ec9dc62963b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:41.587338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 672/1344] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:41.592605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b995a-9974-437a-b141-64cc407b3326 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:41.593598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b995a-9974-437a-b141-64cc407b3326 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:41.593764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b995a-9974-437a-b141-64cc407b3326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:41.621628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-792b995a-9974-437a-b141-64cc407b3326 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:41.622817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-792b995a-9974-437a-b141-64cc407b3326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:41.697813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 673/1345] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:41.735062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21afa9-4535-49b5-9423-6ec9dc62963b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:41.745713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21afa9-4535-49b5-9423-6ec9dc62963b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:41.749733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:41.750383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:41.750662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:41.750965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:41.760312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:41.761758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4c74f-9851-4c0f-9dfd-ad1c9a22512d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:41.772100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 673/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:45.976865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d53ce-3782-4059-b7f3-89775a2ed9dc None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:45.977350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d53ce-3782-4059-b7f3-89775a2ed9dc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:45.977898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d53ce-3782-4059-b7f3-89775a2ed9dc None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:45.978810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 674/1347] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:45.984592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6231-0fa5-4533-b7a9-45082891aaf2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:45.985032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6231-0fa5-4533-b7a9-45082891aaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:45.985125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6231-0fa5-4533-b7a9-45082891aaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:46.003738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1ebe6231-0fa5-4533-b7a9-45082891aaf2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:46.004923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1ebe6231-0fa5-4533-b7a9-45082891aaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:46.012815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 674/1348] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:46.056189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d53ce-3782-4059-b7f3-89775a2ed9dc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:10:46.067364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d53ce-3782-4059-b7f3-89775a2ed9dc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:10:46.070776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:10:46.071433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:46.071712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:46.071981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:46.080950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:10:46.082375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7607964-4a46-4481-bf0e-345ddf6abaa0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:10:46.091636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 675/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:10:51.597607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d320b-99b1-4c14-bccf-f2e4b7fb4bad None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:51.598065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d320b-99b1-4c14-bccf-f2e4b7fb4bad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:51.598156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d320b-99b1-4c14-bccf-f2e4b7fb4bad None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:51.600115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 675/1350] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:51.605429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f1ee2-0ee6-4201-8417-47550fc66525 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:51.606275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f1ee2-0ee6-4201-8417-47550fc66525 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:51.606727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f1ee2-0ee6-4201-8417-47550fc66525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:51.754955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-006f1ee2-0ee6-4201-8417-47550fc66525 None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:51.756222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-006f1ee2-0ee6-4201-8417-47550fc66525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:51.787902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:51.839267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 676/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:10:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:55.709471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efdb71-7c0c-463e-be25-a57d5674b826 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:55.710000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efdb71-7c0c-463e-be25-a57d5674b826 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:55.710452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efdb71-7c0c-463e-be25-a57d5674b826 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:55.712028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 676/1352] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:55.717030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014781-d613-4f05-bdfc-7d2cc82c6ee1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:55.717533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014781-d613-4f05-bdfc-7d2cc82c6ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:55.718588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014781-d613-4f05-bdfc-7d2cc82c6ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:55.738618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-61014781-d613-4f05-bdfc-7d2cc82c6ee1 None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:55.740003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-61014781-d613-4f05-bdfc-7d2cc82c6ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:55.748880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 677/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:10:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:10:56.002764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=72 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:10:57.653451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df26573-983d-49e8-a188-2b4205a17add None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:57.654086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df26573-983d-49e8-a188-2b4205a17add None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:57.654086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df26573-983d-49e8-a188-2b4205a17add None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:57.655526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 677/1354] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:10:57.660426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114eab-8784-4c10-95e8-7ee268431c90 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:10:57.660881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114eab-8784-4c10-95e8-7ee268431c90 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:10:57.661003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114eab-8784-4c10-95e8-7ee268431c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:10:57.679569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7d114eab-8784-4c10-95e8-7ee268431c90 None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:10:57.680762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7d114eab-8784-4c10-95e8-7ee268431c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:10:57.690302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 678/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:10:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:01.496294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf2d45-c629-4361-ab01-2e00d55d83bd None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:01.496859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf2d45-c629-4361-ab01-2e00d55d83bd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:01.496859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf2d45-c629-4361-ab01-2e00d55d83bd None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:01.498200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 678/1356] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:01.503680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d9af-aee2-422c-9139-68f8b4df99c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:01.504067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d9af-aee2-422c-9139-68f8b4df99c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:01.504223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d9af-aee2-422c-9139-68f8b4df99c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:01.524276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f6d5d9af-aee2-422c-9139-68f8b4df99c6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:01.525593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f6d5d9af-aee2-422c-9139-68f8b4df99c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:01.534716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 679/1357] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:01.586754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7da2e-1963-4938-8911-68843da577df None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:01.587042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7da2e-1963-4938-8911-68843da577df None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:01.587367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7da2e-1963-4938-8911-68843da577df None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:01.588707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 679/1358] 10.222.0.39 () {54 vars in 961 bytes} [Thu May 26 03:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:01.596043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533e57-4c89-4a5a-be5d-04f7748f40a5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:01.596331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533e57-4c89-4a5a-be5d-04f7748f40a5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:01.596609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9533e57-4c89-4a5a-be5d-04f7748f40a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:01.653580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b9533e57-4c89-4a5a-be5d-04f7748f40a5 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:01.654944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b9533e57-4c89-4a5a-be5d-04f7748f40a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:01.741597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 680/1359] 10.222.0.39 () {58 vars in 1058 bytes} [Thu May 26 03:11:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:01.749636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7da2e-1963-4938-8911-68843da577df None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:01.764089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7da2e-1963-4938-8911-68843da577df None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:01.769881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:01.770642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:01.771144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:01.771604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:01.781293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:01.783101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d340bb-af16-4121-98dc-a3c40d5a4157 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:01.799019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 680/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:04.264189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898687-d729-4af0-b4f5-0df322282e7b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:04.265870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898687-d729-4af0-b4f5-0df322282e7b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:04.265870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31898687-d729-4af0-b4f5-0df322282e7b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:04.267220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 681/1361] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:04.273853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102ba7-0a88-4ade-9241-d46ba475dc58 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:04.274323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102ba7-0a88-4ade-9241-d46ba475dc58 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:04.274429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f102ba7-0a88-4ade-9241-d46ba475dc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:04.293398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7f102ba7-0a88-4ade-9241-d46ba475dc58 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:04.294591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7f102ba7-0a88-4ade-9241-d46ba475dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:04.304077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 681/1362] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:04.348121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31898687-d729-4af0-b4f5-0df322282e7b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:04.363582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31898687-d729-4af0-b4f5-0df322282e7b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:04.374934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:04.375612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:04.375894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:04.376166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:04.385869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:04.387473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c97c5-1935-488c-9a18-4042203ff9f1 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:04.402165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 682/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:06.966896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec418fb-77b9-4ff0-8c34-3fe63e4ab4f8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:06.968043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec418fb-77b9-4ff0-8c34-3fe63e4ab4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:06.968043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec418fb-77b9-4ff0-8c34-3fe63e4ab4f8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:06.969347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 682/1364] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:06.974592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cfe1-ee1d-43f1-947c-a509199dba89 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:06.975030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cfe1-ee1d-43f1-947c-a509199dba89 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:06.975138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964cfe1-ee1d-43f1-947c-a509199dba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:06.994134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c964cfe1-ee1d-43f1-947c-a509199dba89 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:06.995408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c964cfe1-ee1d-43f1-947c-a509199dba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:07.003446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 683/1365] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:07.051871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b72c33-d070-4d51-a15d-61edc5fd28fa None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:07.052080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b72c33-d070-4d51-a15d-61edc5fd28fa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:07.052363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b72c33-d070-4d51-a15d-61edc5fd28fa None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:07.053725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 683/1366] 10.222.0.39 () {54 vars in 961 bytes} [Thu May 26 03:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:07.059445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1beb44-1452-43cc-b679-6f5abcb7cddf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:07.060023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1beb44-1452-43cc-b679-6f5abcb7cddf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:07.060509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1beb44-1452-43cc-b679-6f5abcb7cddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:07.079764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9a1beb44-1452-43cc-b679-6f5abcb7cddf None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:07.080975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9a1beb44-1452-43cc-b679-6f5abcb7cddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:07.090534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 684/1367] 10.222.0.39 () {58 vars in 1058 bytes} [Thu May 26 03:11:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:07.099693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b72c33-d070-4d51-a15d-61edc5fd28fa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:07.115311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b72c33-d070-4d51-a15d-61edc5fd28fa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:07.122041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:07.122753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:07.123040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:07.123358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:07.133186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:07.135132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96748ce9-faf7-464b-aa24-65a5941a3acd service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:07.146556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 684/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:09.454561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914b84b-bbe1-4b16-8760-bb23df7aa297 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:09.455454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914b84b-bbe1-4b16-8760-bb23df7aa297 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:09.455739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914b84b-bbe1-4b16-8760-bb23df7aa297 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:09.457129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 685/1369] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:09.462687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd5b3-2d4d-4cae-9d62-ab35e74fe30e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:09.463144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd5b3-2d4d-4cae-9d62-ab35e74fe30e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:09.463687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd5b3-2d4d-4cae-9d62-ab35e74fe30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:09.482044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b7fbd5b3-2d4d-4cae-9d62-ab35e74fe30e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:09.483904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b7fbd5b3-2d4d-4cae-9d62-ab35e74fe30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:09.492461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 685/1370] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:09.537382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914b84b-bbe1-4b16-8760-bb23df7aa297 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:09.553038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914b84b-bbe1-4b16-8760-bb23df7aa297 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:09.558516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:09.559233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:09.559612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:09.559962 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:09.569192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:09.570927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0844f-cf79-4a11-9727-59d65269ca01 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:09.583089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 686/1371] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:09.777210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38f5-d25b-4caf-9047-c3198e5a12ae None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:09.777751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38f5-d25b-4caf-9047-c3198e5a12ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:09.778207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38f5-d25b-4caf-9047-c3198e5a12ae None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:09.780325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 686/1372] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:09.786611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986fb6-c81f-4aed-8f96-8a167ec4135d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:09.786992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986fb6-c81f-4aed-8f96-8a167ec4135d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:09.787207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f986fb6-c81f-4aed-8f96-8a167ec4135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:09.807740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8f986fb6-c81f-4aed-8f96-8a167ec4135d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:09.809414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8f986fb6-c81f-4aed-8f96-8a167ec4135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:09.819302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 687/1373] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:10.129751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb0a2-67aa-48ed-8acc-e7efaf92a507 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:10.129751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb0a2-67aa-48ed-8acc-e7efaf92a507 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:10.129751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb0a2-67aa-48ed-8acc-e7efaf92a507 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:10.130415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 687/1374] 10.222.0.39 () {54 vars in 963 bytes} [Thu May 26 03:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:10.136971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7bba2-4b4e-4421-b5ab-1063cbea8a10 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:10.138732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7bba2-4b4e-4421-b5ab-1063cbea8a10 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:10.138732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7bba2-4b4e-4421-b5ab-1063cbea8a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:10.146649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:11:10.192528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c3a7bba2-4b4e-4421-b5ab-1063cbea8a10 None None] MFA Rules not processed for user `5524a8ebd91f4e5eb7311923401c7798`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:10.194045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c3a7bba2-4b4e-4421-b5ab-1063cbea8a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:10.258558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 688/1375] 10.222.0.39 () {58 vars in 1060 bytes} [Thu May 26 03:11:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:10.266562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14eb0a2-67aa-48ed-8acc-e7efaf92a507 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:10.277589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14eb0a2-67aa-48ed-8acc-e7efaf92a507 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:10.281953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:10.283403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:10.283748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:10.284227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:10.296755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:10.298205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6d189-b9b3-4915-aef6-167d75723ba4 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:10.308384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 688/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:11.877165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321956b-c6e0-4260-bddd-73f9e6739593 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:11.878098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321956b-c6e0-4260-bddd-73f9e6739593 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:11.878380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321956b-c6e0-4260-bddd-73f9e6739593 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:11.880016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 689/1377] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:11.885979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d4b2-0da4-4d61-9fa4-2d366d84d758 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:11.886563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d4b2-0da4-4d61-9fa4-2d366d84d758 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:11.886563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d4b2-0da4-4d61-9fa4-2d366d84d758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:11.906037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ce03d4b2-0da4-4d61-9fa4-2d366d84d758 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:11.907319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ce03d4b2-0da4-4d61-9fa4-2d366d84d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:11.915544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 689/1378] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:11.962325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321956b-c6e0-4260-bddd-73f9e6739593 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:11.977878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321956b-c6e0-4260-bddd-73f9e6739593 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:11.984756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:11.985417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:11.985704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:11.985988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:11.995466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:11.997115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee4ace-089a-42c7-ab3d-c1af9efbd13d service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:12.008952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 690/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:14.280381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ee8-1c2c-4c4a-bb7a-3518add04ec1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:14.280906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ee8-1c2c-4c4a-bb7a-3518add04ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:14.280906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ee8-1c2c-4c4a-bb7a-3518add04ec1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:14.282297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 690/1380] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:14.287551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095144c1-32ea-46fc-b3f2-887a0eaa5cef None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:14.288364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095144c1-32ea-46fc-b3f2-887a0eaa5cef None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:14.288646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095144c1-32ea-46fc-b3f2-887a0eaa5cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:14.313498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-095144c1-32ea-46fc-b3f2-887a0eaa5cef None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:14.314698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-095144c1-32ea-46fc-b3f2-887a0eaa5cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:14.322615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 691/1381] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:14.371817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2ee8-1c2c-4c4a-bb7a-3518add04ec1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:14.387432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2ee8-1c2c-4c4a-bb7a-3518add04ec1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:14.393978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:14.394617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:14.394899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:14.395171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:14.404157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:14.405736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328bd87-1f93-4144-b0e3-f0f737834760 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:14.416046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 691/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:16.701865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275c5f-d1d3-4a28-873b-c28faa0b4f62 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:16.702489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275c5f-d1d3-4a28-873b-c28faa0b4f62 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:16.702489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34275c5f-d1d3-4a28-873b-c28faa0b4f62 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:16.703813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 692/1383] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:16.709174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372d6df-5b4c-4484-adb5-e7b0ad835ef1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:16.709603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372d6df-5b4c-4484-adb5-e7b0ad835ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:16.709736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372d6df-5b4c-4484-adb5-e7b0ad835ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:16.728099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7372d6df-5b4c-4484-adb5-e7b0ad835ef1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:16.729299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7372d6df-5b4c-4484-adb5-e7b0ad835ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:16.737637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 692/1384] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:16.779231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275c5f-d1d3-4a28-873b-c28faa0b4f62 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:16.796169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34275c5f-d1d3-4a28-873b-c28faa0b4f62 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:16.801985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:16.802724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:16.803108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:16.803509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:16.813213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:16.814740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392346-c7bc-4c8e-bbb9-ba9bb6b421e2 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:16.825718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 693/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:19.080795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777e0e-7e53-443d-8e56-240b3f809ad1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:19.081330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777e0e-7e53-443d-8e56-240b3f809ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:19.081330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86777e0e-7e53-443d-8e56-240b3f809ad1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:19.082727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 693/1386] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:19.088055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fc1d-3e3d-4428-b44c-2d57b4f072e1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:19.088454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fc1d-3e3d-4428-b44c-2d57b4f072e1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:19.088593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8fc1d-3e3d-4428-b44c-2d57b4f072e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:19.106927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-32d8fc1d-3e3d-4428-b44c-2d57b4f072e1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:19.108796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-32d8fc1d-3e3d-4428-b44c-2d57b4f072e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:19.116739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 694/1387] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:19.162164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86777e0e-7e53-443d-8e56-240b3f809ad1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:19.184270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86777e0e-7e53-443d-8e56-240b3f809ad1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:19.192837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:19.193800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:19.194206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:19.194607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:19.208710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:19.210344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b957d-f110-4275-9e04-becf5aa97a20 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:19.220360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 694/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:21.477214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5092-06c5-4746-b56a-d70da211b7c4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:21.477810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5092-06c5-4746-b56a-d70da211b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:21.477810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab5092-06c5-4746-b56a-d70da211b7c4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:21.479125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 695/1389] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:21.484486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340ada-651c-4e3b-ade5-b42f78df6dba None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:21.484934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340ada-651c-4e3b-ade5-b42f78df6dba None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:21.485058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38340ada-651c-4e3b-ade5-b42f78df6dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:21.504345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-38340ada-651c-4e3b-ade5-b42f78df6dba None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:21.505729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-38340ada-651c-4e3b-ade5-b42f78df6dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:21.515312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 695/1390] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:21.562494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab5092-06c5-4746-b56a-d70da211b7c4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:21.578208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab5092-06c5-4746-b56a-d70da211b7c4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:21.584529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:21.585528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:21.585875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:21.586294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:21.596666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:21.598305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8f8b-23ea-4269-993d-38621487c40c service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:21.608028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 696/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:23.898077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89318-9fe1-49e4-b8f3-c968abf56aa8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:23.899047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89318-9fe1-49e4-b8f3-c968abf56aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:23.899047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89318-9fe1-49e4-b8f3-c968abf56aa8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:23.900097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 696/1392] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:23.906609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2098d-4e2c-4800-826a-c4219d75b30e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:23.907284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2098d-4e2c-4800-826a-c4219d75b30e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:23.907284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2098d-4e2c-4800-826a-c4219d75b30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:23.925782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-51f2098d-4e2c-4800-826a-c4219d75b30e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:23.926996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-51f2098d-4e2c-4800-826a-c4219d75b30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:23.935194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 697/1393] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:23.979891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a89318-9fe1-49e4-b8f3-c968abf56aa8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:23.994292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a89318-9fe1-49e4-b8f3-c968abf56aa8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:24.001014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:24.002053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:24.002549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:24.003012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:24.017075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:24.018662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca97c0a-8f12-41f3-8afe-ef1c374e9527 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:24.028263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 697/1394] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:26.081175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ca8b0-8469-411c-beb3-811f83854be8 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.081774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ca8b0-8469-411c-beb3-811f83854be8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.081774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ca8b0-8469-411c-beb3-811f83854be8 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.083145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 698/1395] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:26.089169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89648a2-ec1d-4136-a5e4-49e1ddf9d65f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.089704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89648a2-ec1d-4136-a5e4-49e1ddf9d65f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.089851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89648a2-ec1d-4136-a5e4-49e1ddf9d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.096413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=52 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:11:26.109935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e89648a2-ec1d-4136-a5e4-49e1ddf9d65f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:26.111180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e89648a2-ec1d-4136-a5e4-49e1ddf9d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:26.119078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 698/1396] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:26.175006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac815-b99c-4279-abb8-32ee8153c0be None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.175324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac815-b99c-4279-abb8-32ee8153c0be None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.175603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ac815-b99c-4279-abb8-32ee8153c0be None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.176998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 699/1397] 10.222.0.39 () {54 vars in 962 bytes} [Thu May 26 03:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:26.182013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.182319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.182597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.230273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] MFA Rules not processed for user `7b48b2079573456c93a286b7d799e950`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:26.231500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:26.278685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b33a9-467b-4b5a-a969-b9ef5885a341 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.278975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b33a9-467b-4b5a-a969-b9ef5885a341 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.279266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b33a9-467b-4b5a-a969-b9ef5885a341 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.281235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 700/1398] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:11:26.286257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.286658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.286938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.293715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 699/1399] 10.222.0.39 () {58 vars in 1059 bytes} [Thu May 26 03:11:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:26.301369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:26.304919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:26.306113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:26.308784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc22468-8c03-43d6-88e8-0c6b821bbf3d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:26.312919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:26.313554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.313840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.314116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.314908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 701/1400] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:26.323556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:26.324991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a356d24-99b9-4c2b-ae70-fd3270b30def service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:26.335367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 700/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:11:26.359563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:26.374926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d27fe-ef65-4558-a673-e23b9823a4c2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:26.380393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:26.381040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.381323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.381599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.391419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:26.393035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4209b-e1df-4b27-84b0-1412fc9a8107 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:26.402849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 702/1402] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:26.876174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689ccc-a007-4e81-9ab5-5698fcc20b98 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:26.876470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689ccc-a007-4e81-9ab5-5698fcc20b98 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:26.876750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20689ccc-a007-4e81-9ab5-5698fcc20b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:26.926075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-20689ccc-a007-4e81-9ab5-5698fcc20b98 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:26.927298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-20689ccc-a007-4e81-9ab5-5698fcc20b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:26.995668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 701/1403] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:11:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:27.040577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fff9b-2b48-4c14-bb96-af7c3e95e78a None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:27.040861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fff9b-2b48-4c14-bb96-af7c3e95e78a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:27.041131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fff9b-2b48-4c14-bb96-af7c3e95e78a None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:27.042546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 703/1404] 10.222.0.39 () {54 vars in 942 bytes} [Thu May 26 03:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:27.049430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e886c0-5da0-4a73-a8c7-16a2911606cd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:27.049751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e886c0-5da0-4a73-a8c7-16a2911606cd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:27.050036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e886c0-5da0-4a73-a8c7-16a2911606cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:27.095204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:11:27.098257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-00e886c0-5da0-4a73-a8c7-16a2911606cd None None] MFA Rules not processed for user `e6eb06c62ee340a0b70fbb88e9f3d09b`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:27.100075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-00e886c0-5da0-4a73-a8c7-16a2911606cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:27.174364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 702/1405] 10.222.0.39 () {58 vars in 1039 bytes} [Thu May 26 03:11:27 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:27.183919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fff9b-2b48-4c14-bb96-af7c3e95e78a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:27.197828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fff9b-2b48-4c14-bb96-af7c3e95e78a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:27.202743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:27.203404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:27.203684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:27.203952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:27.211994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:27.213560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6d78a-5c23-4cb6-a604-99500c85a079 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:27.222135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 704/1406] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:11:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:11:28.693975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d38df-6fdc-4cce-9513-cb5132dde281 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:28.694522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d38df-6fdc-4cce-9513-cb5132dde281 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:28.694522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d38df-6fdc-4cce-9513-cb5132dde281 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:28.696206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 703/1407] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:28.701037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccbfe0-dc5f-48a8-947b-fee1e530b713 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:28.701987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccbfe0-dc5f-48a8-947b-fee1e530b713 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:28.702375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccbfe0-dc5f-48a8-947b-fee1e530b713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:28.723192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-beccbfe0-dc5f-48a8-947b-fee1e530b713 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:28.724443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-beccbfe0-dc5f-48a8-947b-fee1e530b713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:28.732824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 705/1408] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:28.781219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d38df-6fdc-4cce-9513-cb5132dde281 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:28.796253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d38df-6fdc-4cce-9513-cb5132dde281 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:28.801868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:28.802506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:28.802788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:28.803069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:28.812558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:28.814140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46b311-8189-4fea-9085-510088f8cbd8 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:28.824728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 704/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:38.714948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:11:38.819579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=36 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:11:50.845061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e835-a35a-4fc5-8bd0-d8d81d90cae9 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:50.845705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e835-a35a-4fc5-8bd0-d8d81d90cae9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:50.845705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e835-a35a-4fc5-8bd0-d8d81d90cae9 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:50.847038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 706/1410] 10.222.0.39 () {54 vars in 918 bytes} [Thu May 26 03:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:50.852760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757df50f-e985-4cf8-9be3-3c5a4d6c8bfb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:50.853304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757df50f-e985-4cf8-9be3-3c5a4d6c8bfb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:50.853405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757df50f-e985-4cf8-9be3-3c5a4d6c8bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:50.879285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-757df50f-e985-4cf8-9be3-3c5a4d6c8bfb None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:50.880495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-757df50f-e985-4cf8-9be3-3c5a4d6c8bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:50.889048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 705/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Thu May 26 03:11:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:50.903683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf7cf-93d7-4abf-8829-bba7162836a4 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:50.903966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf7cf-93d7-4abf-8829-bba7162836a4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:50.904376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf7cf-93d7-4abf-8829-bba7162836a4 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:50.905895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 707/1412] 10.222.0.39 () {54 vars in 918 bytes} [Thu May 26 03:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:50.910524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979394a5-1507-43e4-b5a3-803b35d4e8f3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:50.910814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979394a5-1507-43e4-b5a3-803b35d4e8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:50.911092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979394a5-1507-43e4-b5a3-803b35d4e8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:50.931187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-979394a5-1507-43e4-b5a3-803b35d4e8f3 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:50.932427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-979394a5-1507-43e4-b5a3-803b35d4e8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:50.942278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 706/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Thu May 26 03:11:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:53.643178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757e29-1498-43ac-a257-054c147f4d13 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:53.643784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757e29-1498-43ac-a257-054c147f4d13 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:53.643784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757e29-1498-43ac-a257-054c147f4d13 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:53.645161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:53.650378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8e92-6c42-4967-b710-573f045ef45b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:53.650828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8e92-6c42-4967-b710-573f045ef45b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:53.650942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb8e92-6c42-4967-b710-573f045ef45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:53.670618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c1eb8e92-6c42-4967-b710-573f045ef45b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:53.671784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c1eb8e92-6c42-4967-b710-573f045ef45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:53.679709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:53.723747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf757e29-1498-43ac-a257-054c147f4d13 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:53.738717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf757e29-1498-43ac-a257-054c147f4d13 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:53.744199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:53.744825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:53.745103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:53.745399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:53.753856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:53.755458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa073-8edc-4984-8127-7b0e1253a799 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:53.765713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 709/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:54.745842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceffcb-fc25-47e3-a49e-ba7b210ae5e5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:54.746414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceffcb-fc25-47e3-a49e-ba7b210ae5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:54.746414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ceffcb-fc25-47e3-a49e-ba7b210ae5e5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:54.747833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:54.752799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf260-2d27-4973-b8fc-3c51298254c5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:54.753226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf260-2d27-4973-b8fc-3c51298254c5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:54.753356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf260-2d27-4973-b8fc-3c51298254c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:54.771963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0b8cf260-2d27-4973-b8fc-3c51298254c5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:54.773160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0b8cf260-2d27-4973-b8fc-3c51298254c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:54.781644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:54.958900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceffcb-fc25-47e3-a49e-ba7b210ae5e5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:54.973265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ceffcb-fc25-47e3-a49e-ba7b210ae5e5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:54.978462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:54.979110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:54.979425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:54.979699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:54.987669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:54.989237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6768a-2d5d-4c65-b15a-4e793c207f01 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:54.998981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 709/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:55.922863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818289e-7211-41b3-9980-a75be6fb12dd None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:55.923474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818289e-7211-41b3-9980-a75be6fb12dd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:55.923474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818289e-7211-41b3-9980-a75be6fb12dd None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:55.924880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:55.930036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2343-c3fe-491e-bf2e-1a56a17569eb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:55.930414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2343-c3fe-491e-bf2e-1a56a17569eb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:55.930608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2343-c3fe-491e-bf2e-1a56a17569eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:55.948899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-610d2343-c3fe-491e-bf2e-1a56a17569eb None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:55.950087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-610d2343-c3fe-491e-bf2e-1a56a17569eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:55.957978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:56.110262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818289e-7211-41b3-9980-a75be6fb12dd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:56.125324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818289e-7211-41b3-9980-a75be6fb12dd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:56.130608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:56.131254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:56.131547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:56.131820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:56.140119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:56.141951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aafaec-5c35-4d81-9c98-ad292107e4c5 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:56.151203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 712/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:57.066511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89828-b936-453c-88d7-08fe500ea397 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:57.067085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89828-b936-453c-88d7-08fe500ea397 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:57.067085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89828-b936-453c-88d7-08fe500ea397 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:57.068481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:57.073476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e9939-3c9d-480d-8b1a-640d6f0b8252 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:57.073852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e9939-3c9d-480d-8b1a-640d6f0b8252 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:57.074024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e9939-3c9d-480d-8b1a-640d6f0b8252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:57.092221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-097e9939-3c9d-480d-8b1a-640d6f0b8252 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:57.093415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-097e9939-3c9d-480d-8b1a-640d6f0b8252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:57.101082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:57.253923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89828-b936-453c-88d7-08fe500ea397 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:57.269045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff89828-b936-453c-88d7-08fe500ea397 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:57.274306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:57.275210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:57.275527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:57.275802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:57.284075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:57.285643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af2f6a-a7b3-475c-9776-d872b0823477 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:57.295801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 712/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:58.220155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152e5a-3a58-4b23-a030-b1369e9d3401 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:58.220720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152e5a-3a58-4b23-a030-b1369e9d3401 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:58.220720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5152e5a-3a58-4b23-a030-b1369e9d3401 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:58.222117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:58.227098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ee78-0fbd-457f-8222-b391e2924190 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:58.227551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ee78-0fbd-457f-8222-b391e2924190 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:58.227666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447ee78-0fbd-457f-8222-b391e2924190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:58.245789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b447ee78-0fbd-457f-8222-b391e2924190 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:58.247037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b447ee78-0fbd-457f-8222-b391e2924190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:58.255392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:11:58.404675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152e5a-3a58-4b23-a030-b1369e9d3401 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:58.418643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5152e5a-3a58-4b23-a030-b1369e9d3401 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:58.424462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:58.425086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:58.425371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:58.425645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:58.433738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:58.436211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf57df8-f3f3-463a-b1df-d22a0c6d8e6b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:58.445652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 715/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:11:59.301886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecf2f-36ab-408a-8c4a-2a8bcaff73b3 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:59.302440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecf2f-36ab-408a-8c4a-2a8bcaff73b3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:59.302440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cecf2f-36ab-408a-8c4a-2a8bcaff73b3 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:59.303851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:11:59.308933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbb3a5-ce33-461e-bd77-7b9d2b51f556 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:59.309391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbb3a5-ce33-461e-bd77-7b9d2b51f556 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:59.309488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbb3a5-ce33-461e-bd77-7b9d2b51f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:59.328376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4cfbb3a5-ce33-461e-bd77-7b9d2b51f556 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:11:59.329567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4cfbb3a5-ce33-461e-bd77-7b9d2b51f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:11:59.337181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:11:59.486647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cecf2f-36ab-408a-8c4a-2a8bcaff73b3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:11:59.501882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cecf2f-36ab-408a-8c4a-2a8bcaff73b3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:11:59.507168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:11:59.507820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:11:59.508099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:11:59.508370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:11:59.517585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:11:59.519151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b49b7-a2d7-498f-ae24-a167f4c7b77d service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:11:59.528322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 715/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:11:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:00.384363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399f7c-8a77-4881-8cfa-0f0e37e561de None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:00.384799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399f7c-8a77-4881-8cfa-0f0e37e561de None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:00.384896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0399f7c-8a77-4881-8cfa-0f0e37e561de None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:00.386284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:00.392079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be05e-fb72-46ee-b46c-8db592199750 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:00.392516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be05e-fb72-46ee-b46c-8db592199750 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:00.392624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be05e-fb72-46ee-b46c-8db592199750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:00.410354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-087be05e-fb72-46ee-b46c-8db592199750 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:00.411578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-087be05e-fb72-46ee-b46c-8db592199750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:00.419207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:00.571765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0399f7c-8a77-4881-8cfa-0f0e37e561de None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:00.586471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0399f7c-8a77-4881-8cfa-0f0e37e561de None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:00.592719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:00.593347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:00.593626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:00.593900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:00.601990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:00.603601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d29f54-4755-421d-8285-4af4257adcc5 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:00.612719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 718/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:12:01.478309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc30af-f87d-4609-8d12-e3f1d1cf1449 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:01.478818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc30af-f87d-4609-8d12-e3f1d1cf1449 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:01.478929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc30af-f87d-4609-8d12-e3f1d1cf1449 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:01.480332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:01.485545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced45ff-3951-48ae-b6fa-2ce7df244e65 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:01.486002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced45ff-3951-48ae-b6fa-2ce7df244e65 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:01.486115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced45ff-3951-48ae-b6fa-2ce7df244e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:01.504214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7ced45ff-3951-48ae-b6fa-2ce7df244e65 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:01.505421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7ced45ff-3951-48ae-b6fa-2ce7df244e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:01.513808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:01.670278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc30af-f87d-4609-8d12-e3f1d1cf1449 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:01.684269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc30af-f87d-4609-8d12-e3f1d1cf1449 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:01.690019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:01.690645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:01.690923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:01.691223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:01.699078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:01.701327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8283a8f-26d0-4372-8d04-040b5595d4d9 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:01.711032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 718/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:02.589260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da00ef-9322-44fc-98e7-48c8bd7c9719 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:02.589796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da00ef-9322-44fc-98e7-48c8bd7c9719 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:02.589796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da00ef-9322-44fc-98e7-48c8bd7c9719 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:02.591178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:02.596050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8093-a4ae-4061-bacf-fa74d247b36a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:02.596456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8093-a4ae-4061-bacf-fa74d247b36a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:02.596614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c8093-a4ae-4061-bacf-fa74d247b36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:02.614253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-380c8093-a4ae-4061-bacf-fa74d247b36a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:02.615479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-380c8093-a4ae-4061-bacf-fa74d247b36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:02.623770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:02.774065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da00ef-9322-44fc-98e7-48c8bd7c9719 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:02.788584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da00ef-9322-44fc-98e7-48c8bd7c9719 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:02.793822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:02.794554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:02.794822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:02.795093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:02.803435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:02.805065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf242fd-808c-45fd-b0d3-bbba466fca25 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:02.814319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 721/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:04.910668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862228-4a85-4664-b249-e8ff28038e9d None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:04.911212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862228-4a85-4664-b249-e8ff28038e9d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:04.911362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89862228-4a85-4664-b249-e8ff28038e9d None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:04.912633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:04.917961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232c51-3a11-45ed-94c6-1eb140a00931 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:04.918396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232c51-3a11-45ed-94c6-1eb140a00931 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:04.918517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232c51-3a11-45ed-94c6-1eb140a00931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:04.936817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-be232c51-3a11-45ed-94c6-1eb140a00931 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:04.938007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-be232c51-3a11-45ed-94c6-1eb140a00931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:04.945637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:04.984399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89862228-4a85-4664-b249-e8ff28038e9d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:04.995383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89862228-4a85-4664-b249-e8ff28038e9d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:04.999754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:05.000391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:05.000671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:05.000969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:05.009722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:05.011147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f7a5c-c749-4192-ba81-542eee6943c2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:05.021522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 721/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:07.407994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe70b-0dd2-47ff-8b76-68bea48e5f72 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:07.408530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe70b-0dd2-47ff-8b76-68bea48e5f72 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:07.408530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe70b-0dd2-47ff-8b76-68bea48e5f72 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:07.409916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:07.415111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a383c76-70ff-4083-b14f-96da5ccf8261 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:07.415545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a383c76-70ff-4083-b14f-96da5ccf8261 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:07.415706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a383c76-70ff-4083-b14f-96da5ccf8261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:07.433852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7a383c76-70ff-4083-b14f-96da5ccf8261 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:07.435801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7a383c76-70ff-4083-b14f-96da5ccf8261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:07.444501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:07.482624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe70b-0dd2-47ff-8b76-68bea48e5f72 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:07.493210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe70b-0dd2-47ff-8b76-68bea48e5f72 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:07.496664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:07.497942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:07.498330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:07.498787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:07.507663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:07.509094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bda39-5c09-425f-bfc7-80dc16c7ecef service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:07.518776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 724/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:12:10.011935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:10.012501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:10.012501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:10.014475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:10.020052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ead286-0bdb-48f0-b6ed-b9cf5b56cd38 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:10.020456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ead286-0bdb-48f0-b6ed-b9cf5b56cd38 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:10.020597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ead286-0bdb-48f0-b6ed-b9cf5b56cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:10.038636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-28ead286-0bdb-48f0-b6ed-b9cf5b56cd38 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:10.039853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-28ead286-0bdb-48f0-b6ed-b9cf5b56cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:10.048076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:10.085722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:10.087462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:10.100664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816c8bf-cad7-4060-a0b7-e91d078ddaea None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:10.104357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:10.104987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:10.105266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:10.105539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:10.114975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:10.116423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dda84c-ac8f-41ca-902b-6a539589013b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:10.125814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 724/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:12.524866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fcb9-2c55-44d6-aa34-edc8398af6ce None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:12.525460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fcb9-2c55-44d6-aa34-edc8398af6ce None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:12.525460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3fcb9-2c55-44d6-aa34-edc8398af6ce None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:12.526791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:12.532268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec020d6-a743-43e0-b924-10f0c1bcc151 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:12.532708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec020d6-a743-43e0-b924-10f0c1bcc151 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:12.532806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec020d6-a743-43e0-b924-10f0c1bcc151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:12.550548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-eec020d6-a743-43e0-b924-10f0c1bcc151 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:12.551763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-eec020d6-a743-43e0-b924-10f0c1bcc151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:12.559391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:12.597766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3fcb9-2c55-44d6-aa34-edc8398af6ce None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:12.608933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3fcb9-2c55-44d6-aa34-edc8398af6ce None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:12.612373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:12.612995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:12.613308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:12.613581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:12.622924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:12.624326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746bf72-69b6-4a1b-a274-a719ee22bbb3 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:12.633496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 727/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:14.893918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8899d-08f4-48be-a861-28c47a68ff63 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:14.894490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8899d-08f4-48be-a861-28c47a68ff63 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:14.894490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8899d-08f4-48be-a861-28c47a68ff63 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:14.895871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:14.901040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395d10-be23-43ef-95b7-7bbee8e6c09f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:14.901452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395d10-be23-43ef-95b7-7bbee8e6c09f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:14.901591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395d10-be23-43ef-95b7-7bbee8e6c09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:14.919462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b6395d10-be23-43ef-95b7-7bbee8e6c09f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:14.920643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b6395d10-be23-43ef-95b7-7bbee8e6c09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:14.928296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:14.965131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8899d-08f4-48be-a861-28c47a68ff63 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:14.976747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8899d-08f4-48be-a861-28c47a68ff63 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:14.980327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:14.980985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:14.981266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:14.981686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:14.991535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:14.992962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e77b4-50c5-4e6c-b06f-be2fcc637a63 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:15.002411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 727/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:17.348057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a81da3-fcfd-437e-81fa-a4fe5334d21f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:17.348601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a81da3-fcfd-437e-81fa-a4fe5334d21f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:17.348601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a81da3-fcfd-437e-81fa-a4fe5334d21f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:17.350012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:17.355389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d955-3e89-4889-82d3-0ed7dc78c452 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:17.355828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d955-3e89-4889-82d3-0ed7dc78c452 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:17.355970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d955-3e89-4889-82d3-0ed7dc78c452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:17.375450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e503d955-3e89-4889-82d3-0ed7dc78c452 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:17.376630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e503d955-3e89-4889-82d3-0ed7dc78c452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:17.384231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:17.423086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a81da3-fcfd-437e-81fa-a4fe5334d21f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:17.433515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a81da3-fcfd-437e-81fa-a4fe5334d21f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:17.437576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:17.438577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:17.438902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:17.439287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:17.449312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:17.452471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76272667-efc6-4182-b0ef-7b1854960c16 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:17.463431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 730/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:19.725436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346919-7b0c-4fcf-b9ad-76b0675b0cc6 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:19.725998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346919-7b0c-4fcf-b9ad-76b0675b0cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:19.725998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346919-7b0c-4fcf-b9ad-76b0675b0cc6 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:19.727404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:19.733325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb09ef6-e3cb-4f78-9a59-0ae5f6236b8f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:19.733722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb09ef6-e3cb-4f78-9a59-0ae5f6236b8f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:19.733879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb09ef6-e3cb-4f78-9a59-0ae5f6236b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:19.739879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=70 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:12:19.753234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ebb09ef6-e3cb-4f78-9a59-0ae5f6236b8f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:19.754427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ebb09ef6-e3cb-4f78-9a59-0ae5f6236b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:19.762090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:19.799925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346919-7b0c-4fcf-b9ad-76b0675b0cc6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:19.805357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=73 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:12:19.812017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346919-7b0c-4fcf-b9ad-76b0675b0cc6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:19.815533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:19.816167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:19.816446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:19.816718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:19.825437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:19.826857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190f934-4a9e-4d8b-9d5c-0dc946e7da23 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:19.836064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 730/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:22.123873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcefc-b36a-4598-a6de-80d6efd80fc8 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:22.124419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcefc-b36a-4598-a6de-80d6efd80fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:22.124419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfcefc-b36a-4598-a6de-80d6efd80fc8 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:22.125809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:22.131720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70063-523c-4bf2-a473-9632979d6222 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:22.132264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70063-523c-4bf2-a473-9632979d6222 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:22.132264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe70063-523c-4bf2-a473-9632979d6222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:22.152986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dfe70063-523c-4bf2-a473-9632979d6222 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:22.154241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dfe70063-523c-4bf2-a473-9632979d6222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:22.162110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:22.203322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfcefc-b36a-4598-a6de-80d6efd80fc8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:22.215007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfcefc-b36a-4598-a6de-80d6efd80fc8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:22.218649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:22.219308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:22.219590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:22.219864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:22.229222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:22.230670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40baf347-9c32-4135-934c-242ae4232427 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:22.241156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 733/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:24.678668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d04ce-60d6-42f2-81be-8bc148c28e27 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:24.679200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d04ce-60d6-42f2-81be-8bc148c28e27 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:24.679200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d04ce-60d6-42f2-81be-8bc148c28e27 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:24.680609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:12:24.685992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d1d4-c091-4fad-90c1-91d8b514314e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:24.686427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d1d4-c091-4fad-90c1-91d8b514314e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:24.686543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6d1d4-c091-4fad-90c1-91d8b514314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:24.704826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fea6d1d4-c091-4fad-90c1-91d8b514314e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:24.706033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fea6d1d4-c091-4fad-90c1-91d8b514314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:24.713754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:24.749992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d04ce-60d6-42f2-81be-8bc148c28e27 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:24.760421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d04ce-60d6-42f2-81be-8bc148c28e27 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:24.764432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:24.765086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:24.765366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:24.765637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:24.774519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:24.775977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ddc65-8fde-47fc-87fe-e9fe00843c27 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:24.785699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 733/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:27.018298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510a2b-1df4-40d5-a094-834db7759a0f None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:27.018829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510a2b-1df4-40d5-a094-834db7759a0f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:27.018829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510a2b-1df4-40d5-a094-834db7759a0f None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:27.020254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:12:27.025535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9e24-bda0-4c84-8d88-d2f82b46d90f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:27.025961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9e24-bda0-4c84-8d88-d2f82b46d90f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:27.026070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb9e24-bda0-4c84-8d88-d2f82b46d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:27.044139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3acb9e24-bda0-4c84-8d88-d2f82b46d90f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:12:27.045321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3acb9e24-bda0-4c84-8d88-d2f82b46d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:12:27.053122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:12:27.089838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a510a2b-1df4-40d5-a094-834db7759a0f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:12:27.100546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a510a2b-1df4-40d5-a094-834db7759a0f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:12:27.104021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:12:27.104648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:12:27.104929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:12:27.105206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:12:27.114176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:12:27.115575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322bad1-c77f-4173-9d1c-29dcd3b8e3e6 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:12:27.125426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 736/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:12:37.039021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=17 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:12:37.120630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=17 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:13:00.504415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777314-1fda-42d8-b231-5e96daf4e3b2 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:00.505035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777314-1fda-42d8-b231-5e96daf4e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:00.505035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777314-1fda-42d8-b231-5e96daf4e3b2 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:00.506333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 735/1471] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:00.511997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e45cd-f557-4ee6-87e1-76e7d5a2aa03 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:00.512614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e45cd-f557-4ee6-87e1-76e7d5a2aa03 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:00.512721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e45cd-f557-4ee6-87e1-76e7d5a2aa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:00.531025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7b4e45cd-f557-4ee6-87e1-76e7d5a2aa03 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:00.532276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b4e45cd-f557-4ee6-87e1-76e7d5a2aa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:00.539944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 737/1472] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:00.586186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb5f-5863-4d51-a155-13bec045bdcf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:00.586472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb5f-5863-4d51-a155-13bec045bdcf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:00.586775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfbb5f-5863-4d51-a155-13bec045bdcf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:00.588188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 736/1473] 10.222.0.39 () {54 vars in 962 bytes} [Thu May 26 03:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:00.593671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda089b9-b417-4b5c-8a7e-5e9823e9fe02 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:00.593958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda089b9-b417-4b5c-8a7e-5e9823e9fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:00.594236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda089b9-b417-4b5c-8a7e-5e9823e9fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:00.612311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-eda089b9-b417-4b5c-8a7e-5e9823e9fe02 None None] MFA Rules not processed for user `7b48b2079573456c93a286b7d799e950`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:00.613502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-eda089b9-b417-4b5c-8a7e-5e9823e9fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:00.619873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 738/1474] 10.222.0.39 () {58 vars in 1059 bytes} [Thu May 26 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:00.626954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfbb5f-5863-4d51-a155-13bec045bdcf None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:00.637829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfbb5f-5863-4d51-a155-13bec045bdcf None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:00.641288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:00.642159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:00.642440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:00.642714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:00.651410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:00.653511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dff1e-9836-405c-87e2-37a6e6b8bb1b service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:00.662684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 737/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:02.980724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaf280-0dae-4b43-9f92-30857e51a575 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:02.981262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaf280-0dae-4b43-9f92-30857e51a575 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:02.981262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaf280-0dae-4b43-9f92-30857e51a575 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:02.982665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 739/1476] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:02.988084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887a614-556b-4d9c-912e-60ee1acdec3e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:02.988534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887a614-556b-4d9c-912e-60ee1acdec3e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:02.988638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887a614-556b-4d9c-912e-60ee1acdec3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:03.006716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4887a614-556b-4d9c-912e-60ee1acdec3e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:03.007941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4887a614-556b-4d9c-912e-60ee1acdec3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:03.015697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 738/1477] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:03.039676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaf280-0dae-4b43-9f92-30857e51a575 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:03.052503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaf280-0dae-4b43-9f92-30857e51a575 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:03.057784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:03.059372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:03.059657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:03.059935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:03.060674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:03.063268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:13:03.063268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:13:03.063268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:13:03.064425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965667a5-5f83-4b9d-8e44-da5f55741d4a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:03.089392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 740/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Thu May 26 03:13:03 2022] GET /identity/v3/projects => generated 2553 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:13:05.330789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a809-e982-49af-af9f-1e3a0087833c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:05.331349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a809-e982-49af-af9f-1e3a0087833c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:05.331455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450a809-e982-49af-af9f-1e3a0087833c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:05.332780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 739/1479] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:05.338116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8470-c7fe-4ab9-83ad-36b6738619ee None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:05.338538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8470-c7fe-4ab9-83ad-36b6738619ee None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:05.338677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8470-c7fe-4ab9-83ad-36b6738619ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:05.356796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a8ac8470-c7fe-4ab9-83ad-36b6738619ee None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:05.357989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a8ac8470-c7fe-4ab9-83ad-36b6738619ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:05.365751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 741/1480] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:05.409955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450a809-e982-49af-af9f-1e3a0087833c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:05.424739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450a809-e982-49af-af9f-1e3a0087833c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:05.430089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:05.430718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:05.431001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:05.431307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:05.439817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:05.441404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f545fe-1da1-4daa-9da2-8d717b386dee service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:05.451109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 740/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:07.857204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb100-f63b-405a-9abe-1ee091331bb5 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:07.857742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb100-f63b-405a-9abe-1ee091331bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:07.857742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfb100-f63b-405a-9abe-1ee091331bb5 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:07.859152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 742/1482] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:07.864514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682883-fc41-4e37-a989-7bd824c0b53f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:07.864914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682883-fc41-4e37-a989-7bd824c0b53f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:07.865081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a682883-fc41-4e37-a989-7bd824c0b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:07.883966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3a682883-fc41-4e37-a989-7bd824c0b53f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:07.885166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3a682883-fc41-4e37-a989-7bd824c0b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:07.893528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 741/1483] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:07.929331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb100-f63b-405a-9abe-1ee091331bb5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:07.951111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfb100-f63b-405a-9abe-1ee091331bb5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:07.954678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:07.955356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:07.955641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:07.955916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:07.965922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:07.967396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb86c7-d0d4-4f1c-acda-e4eb68d2151f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:07.977745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 743/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:10.211535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78eb22-7e35-4857-982c-3e7b942c47ba None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:10.212109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78eb22-7e35-4857-982c-3e7b942c47ba None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:10.212109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78eb22-7e35-4857-982c-3e7b942c47ba None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:10.213427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 742/1485] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:13:10.219176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59dac-1a98-4623-acd5-e00f45ffac78 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:10.219651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59dac-1a98-4623-acd5-e00f45ffac78 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:10.219766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c59dac-1a98-4623-acd5-e00f45ffac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:10.238015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f0c59dac-1a98-4623-acd5-e00f45ffac78 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:10.239203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f0c59dac-1a98-4623-acd5-e00f45ffac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:10.247504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 744/1486] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:10.284533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78eb22-7e35-4857-982c-3e7b942c47ba None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:10.295829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78eb22-7e35-4857-982c-3e7b942c47ba None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:10.299514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:10.300148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:10.300480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:10.300750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:10.309648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:10.311107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42213a5f-d2e2-470e-ba56-d18248ae3c14 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:10.320349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 743/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:12.604503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579c57e-695f-4c49-ba7b-f4dd92ee0a9e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:12.605316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579c57e-695f-4c49-ba7b-f4dd92ee0a9e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:12.605316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579c57e-695f-4c49-ba7b-f4dd92ee0a9e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:12.606582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 745/1488] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:12.611884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200f89-f746-4b91-ba77-a52b779065f4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:12.612324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200f89-f746-4b91-ba77-a52b779065f4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:12.612460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0200f89-f746-4b91-ba77-a52b779065f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:12.630331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c0200f89-f746-4b91-ba77-a52b779065f4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:12.631554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c0200f89-f746-4b91-ba77-a52b779065f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:12.640180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 744/1489] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:12.679976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579c57e-695f-4c49-ba7b-f4dd92ee0a9e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:12.691085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579c57e-695f-4c49-ba7b-f4dd92ee0a9e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:12.694632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:12.695294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:12.695582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:12.695858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:12.705472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:12.706958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537263e8-72f9-4245-9c28-aaaef2fa62ac service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:12.716349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 746/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:15.169197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f93d9-4d03-4cc7-a0f3-a4c613ad0fb1 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:15.169757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f93d9-4d03-4cc7-a0f3-a4c613ad0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:15.169757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f93d9-4d03-4cc7-a0f3-a4c613ad0fb1 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:15.171080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 745/1491] 10.222.0.39 () {54 vars in 926 bytes} [Thu May 26 03:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:13:15.176275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33db6d4-8118-4f76-880c-12fe82aa8489 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:15.176698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33db6d4-8118-4f76-880c-12fe82aa8489 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:15.176836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33db6d4-8118-4f76-880c-12fe82aa8489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:15.195111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a33db6d4-8118-4f76-880c-12fe82aa8489 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:13:15.196341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a33db6d4-8118-4f76-880c-12fe82aa8489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:13:15.204210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 747/1492] 10.222.0.39 () {58 vars in 1023 bytes} [Thu May 26 03:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:13:15.243073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f93d9-4d03-4cc7-a0f3-a4c613ad0fb1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:13:15.254242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f93d9-4d03-4cc7-a0f3-a4c613ad0fb1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:13:15.257669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:13:15.258294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:13:15.258572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:13:15.258843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:13:15.268251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:13:15.269673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0822c-2f29-4ccc-8e60-fb9c191256ac service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:13:15.279119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 746/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:13:25.190060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=33 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:13:25.274794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=33 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:15:23.952575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:23.953165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:23.953165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:23.970953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:15:23.972187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:23.980743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 748/1494] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:15:23.989922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:23.990426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:23.990530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.008841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:15:24.010151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:24.019657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 747/1495] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:15:24.024813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.035592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550ff3b-6779-438a-a967-2daf00a6fc24 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.040098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.040752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.041032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.041814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.042543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:15:24.042984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.044582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.044582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:15:24.044582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:15:24.045473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.058378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 749/1496] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:15:24.066916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.076252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24908e20-ac5d-47b2-8f4c-249d64e366ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.080762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.081410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.081695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.081974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.082655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.084467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.084467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:15:24.084467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:15:24.085674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.127309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 748/1497] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:15:24 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:15:24.132682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.145554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db344214-a2cd-497c-95de-7b7cded129d5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.215763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.216356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.216651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.216905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.217693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:15:24.218636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:15:24.220005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.241592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 750/1498] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:15:24 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:15:24.246587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.257398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006a65d-1776-4384-b16c-2f973ad3af47 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.262308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.263325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.263609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.263889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.264606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.265626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.265626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:15:24.265626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:15:24.266493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.280840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 749/1499] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:15:24.285381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.298075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6079e21-7614-40d9-a881-cddfd8c0cc98 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.306045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.306701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.308023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.308370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.309101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.310181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.310181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:15:24.310181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:15:24.311061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f13dc3-a2d9-495a-97d5-5973a1d09635 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.327653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 751/1500] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:15:24.332138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.342780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3f02-7c14-4edd-be59-9475788ac4bd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.347408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.348290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.348575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.348849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] PATH_INFO: `/v3/projects/f1347dd6bba94f93832121e1eaec2b35/users/5d65379ca4704e0098e49e6d05aac899/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.364849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1347dd6bba94f93832121e1eaec2b35, user_id=5d65379ca4704e0098e49e6d05aac899, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.366103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.366103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:15:24.366103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:15:24.368120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18756c4e-0c5f-47ad-a110-5beeccea2589 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.381817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 750/1501] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:15:24 2022] PUT /identity/v3/projects/f1347dd6bba94f93832121e1eaec2b35/users/5d65379ca4704e0098e49e6d05aac899/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:15:24.394054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa70f-6295-47a9-a9b2-754a5deb3e4d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.394503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa70f-6295-47a9-a9b2-754a5deb3e4d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.395005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa70f-6295-47a9-a9b2-754a5deb3e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.427712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7c2aa70f-6295-47a9-a9b2-754a5deb3e4d None None] MFA Rules not processed for user `5d65379ca4704e0098e49e6d05aac899`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:15:24.428934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c2aa70f-6295-47a9-a9b2-754a5deb3e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:24.499097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 752/1502] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:15:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:15:24.512622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb75be8-36dd-403c-8440-8d95f8974059 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.512912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb75be8-36dd-403c-8440-8d95f8974059 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.513187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb75be8-36dd-403c-8440-8d95f8974059 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.514576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 751/1503] 10.222.0.39 () {54 vars in 963 bytes} [Thu May 26 03:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 03:15:24.519768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480491-7c57-4cef-89a6-3246685f7ea9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.520052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480491-7c57-4cef-89a6-3246685f7ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.520330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480491-7c57-4cef-89a6-3246685f7ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.539072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-73480491-7c57-4cef-89a6-3246685f7ea9 None None] MFA Rules not processed for user `5524a8ebd91f4e5eb7311923401c7798`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:15:24.540319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-73480491-7c57-4cef-89a6-3246685f7ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:24.585253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:15:24.595046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 753/1504] 10.222.0.39 () {58 vars in 1060 bytes} [Thu May 26 03:15:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:15:24.602764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb75be8-36dd-403c-8440-8d95f8974059 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.615535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb75be8-36dd-403c-8440-8d95f8974059 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.619154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.620147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.620430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.620707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.630195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.631684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.647901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 752/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:15:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:15:24.693757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64904de-3bd5-4da3-9027-788340426a6b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.694046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64904de-3bd5-4da3-9027-788340426a6b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.694320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64904de-3bd5-4da3-9027-788340426a6b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.695642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 754/1506] 10.222.0.39 () {52 vars in 829 bytes} [Thu May 26 03:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:15:24.699985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.765626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacc904-821f-44e7-a875-0b697dd45938 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:24.770301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:24.771167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.771667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.771954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] PATH_INFO: `/v3/users/5d65379ca4704e0098e49e6d05aac899` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.773132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d65379ca4704e0098e49e6d05aac899)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:24.774147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:24.774147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:15:24.774147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:15:24.775511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:24.844936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d65379ca4704e0098e49e6d05aac899'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:24.857187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d65379ca4704e0098e49e6d05aac899'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:24.857958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d65379ca4704e0098e49e6d05aac899'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:24.867228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d65379ca4704e0098e49e6d05aac899'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:24.867911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d65379ca4704e0098e49e6d05aac899'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:24.884275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:15:24.895319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 753/1507] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:15:24 2022] DELETE /identity/v3/users/5d65379ca4704e0098e49e6d05aac899 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:15:24.901269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:24.901638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:24.901910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:24.918812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:15:24.920099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:24.977065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 755/1508] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:15:24.987017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:24.988434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:25.044546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3a7b9-e468-4b57-a674-3ea2ce43862f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:25.048063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:25.048700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:25.048979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:25.049251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:25.058704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:25.060160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be09888-9d1c-4b8e-9b31-41d2e0b41fb7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:25.069229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 754/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:15:25.425942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:15:25.428108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:15:25.440854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633177f-ceed-49cc-acab-98ccc76af670 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:15:25.445823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:15:25.446696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:15:25.447028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:15:25.447334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] PATH_INFO: `/v3/projects/f1347dd6bba94f93832121e1eaec2b35` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:15:25.449155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1347dd6bba94f93832121e1eaec2b35)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:15:25.450236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:15:25.450236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:15:25.450236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:15:25.451464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:15:25.512281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1347dd6bba94f93832121e1eaec2b35', 'request_id': 'req-57984697-e2bd-4007-bab6-49bdd9b46330'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:25.526766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1347dd6bba94f93832121e1eaec2b35', 'request_id': 'req-57984697-e2bd-4007-bab6-49bdd9b46330'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:25.536641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1347dd6bba94f93832121e1eaec2b35', 'request_id': 'req-57984697-e2bd-4007-bab6-49bdd9b46330'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:25.547751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] The token cache is being invalidate because project f1347dd6bba94f93832121e1eaec2b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:15:25.548046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:25.548761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-57984697-e2bd-4007-bab6-49bdd9b46330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:15:25.550316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 756/1510] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:15:25 2022] DELETE /identity/v3/projects/f1347dd6bba94f93832121e1eaec2b35 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:15:35.065231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:15:35.544688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:17:04.528008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:04.528595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:04.528595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:04.529937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:04.534895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5bfa-1065-493e-b54a-ae2ced75a459 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:04.535341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5bfa-1065-493e-b54a-ae2ced75a459 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:04.535489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5bfa-1065-493e-b54a-ae2ced75a459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:04.554975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-937f5bfa-1065-493e-b54a-ae2ced75a459 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:04.556225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-937f5bfa-1065-493e-b54a-ae2ced75a459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:04.613172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:17:04.787763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:04.789463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:04.867524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994e768-a599-4648-a99c-3d956b78f1d9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:04.874841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:04.875462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:04.875919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:04.876226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:04.885273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:04.886895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6bd9-391c-4344-b1be-50472f418b9e service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:04.896733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 756/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:05.784767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe1a68-df42-48f4-91a4-816eb3c756ca None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:05.785244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe1a68-df42-48f4-91a4-816eb3c756ca None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:05.785359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe1a68-df42-48f4-91a4-816eb3c756ca None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:05.786690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:05.791888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c670b-4634-4038-999e-9ff60291caf2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:05.792317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c670b-4634-4038-999e-9ff60291caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:05.792476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c670b-4634-4038-999e-9ff60291caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:05.810887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5d0c670b-4634-4038-999e-9ff60291caf2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:05.812107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5d0c670b-4634-4038-999e-9ff60291caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:05.819754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:05.973558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe1a68-df42-48f4-91a4-816eb3c756ca None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:05.988413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe1a68-df42-48f4-91a4-816eb3c756ca None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:05.993829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:05.994430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:05.994835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:05.995113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:06.003009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:06.004692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973769c-d451-4a72-97f3-ad54b1d78755 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:06.015245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 759/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:06.105963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedabe46-ddad-47c6-884c-306f97d2459b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:06.106255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedabe46-ddad-47c6-884c-306f97d2459b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:06.106532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedabe46-ddad-47c6-884c-306f97d2459b None None] PATH_INFO: `/v3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:06.107901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 758/1517] 10.222.0.45 () {54 vars in 925 bytes} [Thu May 26 03:17:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:17:06.113964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561382f8-d243-4e73-973e-91dba5d1c5dd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:06.114248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561382f8-d243-4e73-973e-91dba5d1c5dd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:06.114520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561382f8-d243-4e73-973e-91dba5d1c5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:06.132815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-561382f8-d243-4e73-973e-91dba5d1c5dd None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:06.134053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-561382f8-d243-4e73-973e-91dba5d1c5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:06.195702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 760/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Thu May 26 03:17:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:17:10.024447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c2e03-43e1-4ff6-b8dd-0855fc15c447 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:10.025007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c2e03-43e1-4ff6-b8dd-0855fc15c447 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:10.025104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c2e03-43e1-4ff6-b8dd-0855fc15c447 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:10.026432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 759/1519] 10.222.0.39 () {54 vars in 960 bytes} [Thu May 26 03:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:10.031459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35341-b307-488a-a847-c67efeca2e8b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:10.031898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35341-b307-488a-a847-c67efeca2e8b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:10.032008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e35341-b307-488a-a847-c67efeca2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:10.051654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a2e35341-b307-488a-a847-c67efeca2e8b None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:10.052847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a2e35341-b307-488a-a847-c67efeca2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:10.062040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 761/1520] 10.222.0.39 () {58 vars in 1057 bytes} [Thu May 26 03:17:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:10.069477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c2e03-43e1-4ff6-b8dd-0855fc15c447 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:10.083048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c2e03-43e1-4ff6-b8dd-0855fc15c447 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:10.088021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:10.088643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:10.088927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:10.089201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:10.097264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:10.098831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542e902-5084-445b-b633-3e04fa3fb5e2 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:10.108311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 760/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:17.296369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516697-9559-4abe-87a1-7f13c7473e25 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:17.296910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516697-9559-4abe-87a1-7f13c7473e25 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:17.297028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516697-9559-4abe-87a1-7f13c7473e25 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:17.298345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:17.303143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065837c8-2a5f-4280-b28f-c827bf85860f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:17.303604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065837c8-2a5f-4280-b28f-c827bf85860f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:17.303739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065837c8-2a5f-4280-b28f-c827bf85860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:17.322286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-065837c8-2a5f-4280-b28f-c827bf85860f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:17.323516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-065837c8-2a5f-4280-b28f-c827bf85860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:17.331222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:17.486652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516697-9559-4abe-87a1-7f13c7473e25 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:17.501843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516697-9559-4abe-87a1-7f13c7473e25 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:17.507233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:17.507889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:17.508169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:17.508487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:17.516734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:17.518613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5e2d6-d7b6-45f5-ba6d-3038544e3a0f service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:17.528826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 763/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:18.410008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbc08-e618-4ff2-b711-fa8cee96f34f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:18.410531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbc08-e618-4ff2-b711-fa8cee96f34f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:18.410652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbbc08-e618-4ff2-b711-fa8cee96f34f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:18.411983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:18.416880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c026dc-72d9-4fe0-85b7-a5b063418781 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:18.417289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c026dc-72d9-4fe0-85b7-a5b063418781 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:18.417440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c026dc-72d9-4fe0-85b7-a5b063418781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:18.435545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-94c026dc-72d9-4fe0-85b7-a5b063418781 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:18.436743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94c026dc-72d9-4fe0-85b7-a5b063418781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:18.444470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 03:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:18.597959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbbc08-e618-4ff2-b711-fa8cee96f34f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:18.612832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbbc08-e618-4ff2-b711-fa8cee96f34f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:18.618168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:18.618799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:18.619720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:18.620004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:18.628125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:18.629652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec834f-3e45-44e4-9072-01e5ca29ea3f service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:18.638973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 763/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:22.577086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:22.577640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:22.577640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:22.579020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:22.584121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4410-eb53-46fe-8efc-7a30d5b9d056 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:22.584539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4410-eb53-46fe-8efc-7a30d5b9d056 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:22.584681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4410-eb53-46fe-8efc-7a30d5b9d056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:22.603002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f03e4410-eb53-46fe-8efc-7a30d5b9d056 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:22.604643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f03e4410-eb53-46fe-8efc-7a30d5b9d056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:22.612250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:22.649529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:22.651954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:22.707177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f640-db29-44e7-a7b3-2c50ef2e432e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:22.710690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:22.711351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:22.711636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:22.711914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:22.721175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:22.722606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eddc0d-c102-44ea-bd66-ca1334595f29 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:22.727433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:17:22.732877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 766/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:24.935815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c869-8c96-45ed-aeb0-c54655cdb93b None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:24.936385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c869-8c96-45ed-aeb0-c54655cdb93b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:24.936385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50c869-8c96-45ed-aeb0-c54655cdb93b None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:24.937759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:24.942903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bce35-539b-4670-9d63-a3a47f16610d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:24.943333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bce35-539b-4670-9d63-a3a47f16610d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:24.943493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bce35-539b-4670-9d63-a3a47f16610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:24.961641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-401bce35-539b-4670-9d63-a3a47f16610d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:24.963599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-401bce35-539b-4670-9d63-a3a47f16610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:24.971325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:25.007738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50c869-8c96-45ed-aeb0-c54655cdb93b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:25.021685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50c869-8c96-45ed-aeb0-c54655cdb93b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:25.026808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:25.027490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:25.027797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:25.028054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:25.040410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:25.041842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9e7d0-ad25-41d1-ba57-2ded13d938ef service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:25.051688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 766/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:28.668195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:28.668878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:28.668878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:28.670132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:28.675222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc57e2-4ab0-4e4c-abac-d1c333c00e39 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:28.675686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc57e2-4ab0-4e4c-abac-d1c333c00e39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:28.675791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc57e2-4ab0-4e4c-abac-d1c333c00e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:28.682812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=62 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:17:28.695113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-18fc57e2-4ab0-4e4c-abac-d1c333c00e39 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:28.696341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-18fc57e2-4ab0-4e4c-abac-d1c333c00e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:28.704091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:28.984981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:28.986742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:29.043375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cd879-4d10-47fa-878f-27b6a2e8b710 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:29.046917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:29.047939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:29.048228 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:29.048501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:29.057400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:29.058833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67eeab4-4b04-4cf3-b913-b323ffd5adce service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:29.068045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 769/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:31.271530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:31.272073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:31.272073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:31.273439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 03:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:17:31.279299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c026ff4-e8a6-4fac-bca6-c14175063414 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:31.279771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c026ff4-e8a6-4fac-bca6-c14175063414 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:31.279874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c026ff4-e8a6-4fac-bca6-c14175063414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:31.298642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0c026ff4-e8a6-4fac-bca6-c14175063414 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:17:31.299851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0c026ff4-e8a6-4fac-bca6-c14175063414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:31.308014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 03:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:17:31.576299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:17:31.577967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:17:31.591253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1337fe-5fd3-409d-9ce9-2c84b7b3560c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:17:31.594720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:17:31.595385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:17:31.595669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:17:31.595944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:17:31.604681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:17:31.606104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec114a-8f20-4c2c-b03a-259e8d94880f service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:17:31.615364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 769/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:17:41.293549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=32 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:17:41.611326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=8 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:00.200629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a37c-ef20-40fe-93e1-9a372adace99 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.201775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a37c-ef20-40fe-93e1-9a372adace99 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.202281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a37c-ef20-40fe-93e1-9a372adace99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.228139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-21c0a37c-ef20-40fe-93e1-9a372adace99 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.229746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-21c0a37c-ef20-40fe-93e1-9a372adace99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.233221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2142b9-78e0-4979-ac00-9e43f444d456 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.234160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2142b9-78e0-4979-ac00-9e43f444d456 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.234684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2142b9-78e0-4979-ac00-9e43f444d456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.238992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 771/1540] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.261320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.262095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.262548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.267118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-fd2142b9-78e0-4979-ac00-9e43f444d456 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.268861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fd2142b9-78e0-4979-ac00-9e43f444d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.288997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 770/1541] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:00.289217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.290496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.305987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.306662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.307184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.313175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 772/1542] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:00.319548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.335274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff353020-b6ed-4628-a2a6-2dba86358070 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.345304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.345988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.346270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.346545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.347306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.347999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.351726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.353145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.354622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.372733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 773/1543] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:19:00.373416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 771/1544] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:00.380417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.388438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.397685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ba88-6091-4b90-90e8-d9e9b15ac907 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.405965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f12a4-b92d-46df-942e-79de733236a1 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.415037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.415468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.416121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.416408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.416690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.417105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.417441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.417749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.418054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.419416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b39d9b-0c05-4d0c-9b3f-0fc723edac91 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.419712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.420640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.422793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd2605-99a9-4679-8c16-d11d0ea5a70c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.441233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 774/1545] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:00.442569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.442860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.443140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.473516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 772/1546] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:00.475284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.475804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.476388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.481499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.481499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.505195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.506986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.553095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 775/1547] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.553722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.564344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567ea2-cb1a-4df5-bf8a-e8daa497e10d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.569088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.570442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.570863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.571277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.571960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.573006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:00.573006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:19:00.573006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:19:00.574216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915e237-4bfc-4456-a339-ba79f4d342c1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.587205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 773/1548] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.588067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.597954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebce33-c35a-4ba6-99f3-7a97b3e526ae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.603382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.604010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 776/1549] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:00.604137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.604360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.604616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.605273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.605389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec921-5b5b-4697-bd2e-efd480fd77c8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.605562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec921-5b5b-4697-bd2e-efd480fd77c8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.605842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec921-5b5b-4697-bd2e-efd480fd77c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.606301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:00.606301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:19:00.606301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:19:00.607648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d963347-ad3c-4501-b5cd-53447e5526a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.624172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-416ec921-5b5b-4697-bd2e-efd480fd77c8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.625476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-416ec921-5b5b-4697-bd2e-efd480fd77c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.630739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 774/1550] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/users => generated 284 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:00.630739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62998eae-8b33-4070-ad67-96346a06a274 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.630739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62998eae-8b33-4070-ad67-96346a06a274 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.630739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62998eae-8b33-4070-ad67-96346a06a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.648832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-62998eae-8b33-4070-ad67-96346a06a274 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.650068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-62998eae-8b33-4070-ad67-96346a06a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.694011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 777/1551] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.695413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcefa8c-2bf9-414f-90d4-ecf2a870fba1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.695746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcefa8c-2bf9-414f-90d4-ecf2a870fba1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.696024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcefa8c-2bf9-414f-90d4-ecf2a870fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.714811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-adcefa8c-2bf9-414f-90d4-ecf2a870fba1 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.716176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-adcefa8c-2bf9-414f-90d4-ecf2a870fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.720204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 775/1552] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.721546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5f14-ab22-4727-893d-7b9f90852652 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.721892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5f14-ab22-4727-893d-7b9f90852652 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.722231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e5f14-ab22-4727-893d-7b9f90852652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.731035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 778/1553] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.732263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.732551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.732826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.746678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1e2e5f14-ab22-4727-893d-7b9f90852652 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.747983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e2e5f14-ab22-4727-893d-7b9f90852652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.751629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.752792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.755969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 776/1554] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.759740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.759740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.759740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.760714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 779/1555] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.761463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.772387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bce2a2-ef15-43d3-b710-788262751a74 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.776874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.777211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.777867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.778080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.778188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.778407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.779089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.781063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e08def-d9ce-421c-9383-050596716832 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.785894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 777/1556] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.786659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.797438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba7443-5f77-4757-9341-8a6dcd7372e9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.802133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.802794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.803076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.803267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 780/1557] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:00.803696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.804626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.804739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd1e-8a86-4d04-bc2f-ce5a17b3175a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.805118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd1e-8a86-4d04-bc2f-ce5a17b3175a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.805398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd1e-8a86-4d04-bc2f-ce5a17b3175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.806537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6264f5e-1f22-4c70-8799-4c8e60347251 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.816807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 778/1558] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:00.820549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.820549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.820549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.823776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cbefcd1e-8a86-4d04-bc2f-ce5a17b3175a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.824980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cbefcd1e-8a86-4d04-bc2f-ce5a17b3175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.833483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 781/1559] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.834529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c07-1119-4480-b4d9-571adf9768e9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.834818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c07-1119-4480-b4d9-571adf9768e9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.835094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b907c07-1119-4480-b4d9-571adf9768e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.837567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.838745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.841368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:00.847936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 779/1560] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.852310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.854640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3b907c07-1119-4480-b4d9-571adf9768e9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.855856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3b907c07-1119-4480-b4d9-571adf9768e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.859040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fa7c-8336-4215-aca8-47d678a1932b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.863521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.863764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 782/1561] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.864168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.864454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.864733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.865025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.865310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.865456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.865584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.865765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.866848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:00.866848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:19:00.866848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:19:00.868204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.887655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 780/1562] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:00.887655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.887963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:00.887963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:00.888146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:00.891950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1d9fc-cf7e-4c9b-bd88-7ed1bdef94bd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.894181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 783/1563] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:00.895031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.897628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.898893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.899525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.900118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.900851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.901163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.902766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.904499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510542-ea9c-4079-ba54-68cc0e172f22 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.909150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.909935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.910218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.910488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.911202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.912820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.916129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 781/1564] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:00.917019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.924170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 784/1565] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:00.924975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.930199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1891e8-9f50-4b56-9888-eb713660b74d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.934568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e39689-657a-4550-aded-47f5e266a436 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.934934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.935628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.935914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.936191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.936911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.938504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.939093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.939744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.940020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.940289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.941024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.941329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.942884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.953356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 782/1566] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:00.953356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.961402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 785/1567] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:00.964161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334e42d-6b46-47a4-9dcf-b148c0de1eb1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.970295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.971837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.972373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.972373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.972373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.973185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.973185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.974527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:00.982179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01012983-15b2-441a-bd9a-8e5366ac52fc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:00.986772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:00.987606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:00.987895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:00.988031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 783/1568] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:00.988214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:00.988722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:00.988898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:00.989190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:00.990833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.004710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a1d5-f31c-4dd4-8f81-e81619885ef4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.005846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 786/1569] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:01.006725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.009942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.010589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.010868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.011138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.011875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.013855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.022766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090dc7d6-68ca-4abd-9e4e-b2781d0888a7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.028046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.028682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.028959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.029230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.029955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:01.030286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.032291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.043966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 784/1570] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:00 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.043966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.054761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 787/1571] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:01.054761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.059890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44956d52-7f64-4f42-90ae-a6bd43e20ec4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.102409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a265-1e26-4190-b926-4ed0f26104a1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.106999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.107686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.107964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.108238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.108923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.110827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.117989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.118658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.118941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.119216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.119975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.121602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.131907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 788/1572] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.133112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.142628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 785/1573] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.143170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6c7fc-497a-4d8e-9c1b-c34e3dd08def admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.143831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.199620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.200600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.200888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.201428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] PATH_INFO: `/v3/projects/421d43c3fe944cfdbe747ea49ff7a905/users/3d0e4c436d5545b1bc5727186db9ba48/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.206394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a23729-c18b-4273-8ae3-81f26e1c6c79 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.211007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.211801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.212092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.212369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.213050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.214947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.215256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421d43c3fe944cfdbe747ea49ff7a905, user_id=3d0e4c436d5545b1bc5727186db9ba48, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.216324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:01.216324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:19:01.216324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:19:01.218245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.231960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 789/1574] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:01 2022] PUT /identity/v3/projects/421d43c3fe944cfdbe747ea49ff7a905/users/3d0e4c436d5545b1bc5727186db9ba48/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:01.233094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.243274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 786/1575] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.243274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.259331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:01.260997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:01.306522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592a392-d17f-49c4-8426-36a0aed8abb6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.308230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd4e-d508-4e5a-8ec6-40351dc50182 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.311370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.312103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.312458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.312788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.312906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.313442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.313718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.313837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.313993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.314687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.316003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.316617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.341754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 790/1576] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.342560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.342700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 787/1577] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.343622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.409357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc056c-a13f-4476-82ce-2c5ee0b98889 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.412529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f548b-82cc-4892-a932-5f373d06f621 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.414576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.415170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.415496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.415790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.416573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.417967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.418596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.418719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.418887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.419159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.419872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.421770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.440981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 791/1578] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.441996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.450485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 788/1579] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.451392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.452171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96012467-6e31-4f86-b169-7ea6835cba66 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.509699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.510502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.510779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.511055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] PATH_INFO: `/v3/projects/ae4cce0a69044ebcba5f2b5352d04cbf/users/3cf837e2ab5b4e58844a1023cf5f9237/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.514238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68373e9-9f7a-403c-98e9-7bd1de1eeb70 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.519033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.519717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.520091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.520273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.520958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.522977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.527388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4cce0a69044ebcba5f2b5352d04cbf, user_id=3cf837e2ab5b4e58844a1023cf5f9237, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.530071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb8017-e0d6-4fc6-8183-ec277ed7036a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.544537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 792/1580] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:01 2022] PUT /identity/v3/projects/ae4cce0a69044ebcba5f2b5352d04cbf/users/3cf837e2ab5b4e58844a1023cf5f9237/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:01.545835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.546126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.546401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.548110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 789/1581] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.551718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.561514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832d191-3f52-4833-bf41-e247d6c6f663 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.573037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] MFA Rules not processed for user `3d0e4c436d5545b1bc5727186db9ba48`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:01.573719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:01.574259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:01.578324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:01.619669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.620516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.620920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.621257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.622019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.623978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.642201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 793/1582] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:01.643086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.646880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 790/1583] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.646880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.657461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979ed50-6b2e-4653-9ece-553cd61ef0a5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.657813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564badca-be0f-4aa9-ae20-406cc8242aa9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.661902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.662557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.662688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.662837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.663107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.663365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.664278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.664748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.664888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.665846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.666857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.667992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.688186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 794/1584] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.689648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.691628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 791/1585] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.692420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.701247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914b27d-5389-4779-95eb-d038fb0c08ab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.702295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba2a1-a4f5-49d2-b33a-941bf257ed43 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.705838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.706483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.706762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.707038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.707131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.707800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.707955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.708089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.708360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.709030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.709392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e951a-11b0-4b83-af62-93b8e54d9bb0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.711835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.721731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 795/1586] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.723617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.724001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.724469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.738715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 792/1587] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/users => generated 285 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:01.738715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.742954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:01.744204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:01.747059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384ff21-2b5f-4834-a50c-3fe4e00aecbc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.751699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.752036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 796/1588] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:01.752353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.752652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.752831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.753009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.753734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.755882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.762140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a81ff-de63-4572-acb1-e46ec33d56ee None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.766369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 793/1589] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.767034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.767310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.767687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.767974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.768252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.769037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.770647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.781490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 797/1590] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.782104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.791618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ff193-5a79-43f9-b515-cd889b2be93f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.796176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.796799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.797082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.797358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.798091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.799699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.811085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 798/1591] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.811985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.822595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016032fa-7a76-4930-89bf-05d7f9ce1818 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.826184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0996095-5ab8-4e85-bafb-e71bfbf88b81 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.827172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.827838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.828113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.828384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.829085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.829780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.830418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.830645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.830760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.830957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.840044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.841463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.841591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 799/1592] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.842342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.850814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 794/1593] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:01.852381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.855088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8fb8-48ec-4c12-a2c9-3cb533858a23 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.861054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.861702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.861983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.862254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.862969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.863089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796cca-8001-494d-af25-210f0c4eca02 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.864804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.867784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.868472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.868756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.869036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.869750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.871368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.874936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 800/1594] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.878830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.881975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 795/1595] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.882844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.893224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabd718-9d25-4998-ba3d-515428ae75b3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.898376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.899282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.899583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.899903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.900624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.902253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.918275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 796/1596] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.918275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.928111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5df60f-9d50-4de6-8ce3-aa8fe82d9c1b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.933307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.934110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.934558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.934820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.935681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.937411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.946221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd74707-d87e-4bbe-8d2a-a8edbc3ff5f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.950195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.950963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.954379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.954517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 797/1597] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.954618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.954707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.964199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5234006-69cc-4a09-a03d-ec8674a0fa16 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.965414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.966969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.968917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.969604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.970201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.970469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:01.971213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:01.972889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:01.980206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 801/1598] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:01.981098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.991796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec76e1-2525-428e-badf-8c5be9d96e70 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:01.992731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 798/1599] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:01.994792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:01.997854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:01.998826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:01.999221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:01.999641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.000502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.002432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.009654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba84845-d3db-40ba-8770-97dd303ae77f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.015431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.019504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 802/1600] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:02.019688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.019688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.019867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.019953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] PATH_INFO: `/v3/projects/22ca67d4b5de455cb8dc976e05776cf3/users/8124939d5de3486989a75ba918c3e71a/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.032114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6f35-5daa-4c76-a747-91c47141f73e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.036730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ca67d4b5de455cb8dc976e05776cf3, user_id=8124939d5de3486989a75ba918c3e71a, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.038286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.040161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.040538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.040920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.041433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.042477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.043155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:02.044997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.063282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 803/1601] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:02.063282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.065719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 799/1602] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:01 2022] PUT /identity/v3/projects/22ca67d4b5de455cb8dc976e05776cf3/users/8124939d5de3486989a75ba918c3e71a/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.066981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.072413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3231ff-1768-4b78-b328-803e71832d8e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.081865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:02.160049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.162614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.163614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.164151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] PATH_INFO: `/v3/projects/f3d32df9c64c414d9c93be35820927c9/users/eb6ed6f6c22945e8944ec2414bc2de87/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.166481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0065bc-c16a-4927-a204-0e912e305106 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.172109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.173004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.173288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.173703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] PATH_INFO: `/v3/projects/0b79dbf6a6ff44b2852078fbc048d413/users/23fc6aa882cd401d873519580c5b52ff/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.185174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d32df9c64c414d9c93be35820927c9, user_id=eb6ed6f6c22945e8944ec2414bc2de87, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.189131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.196983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b79dbf6a6ff44b2852078fbc048d413, user_id=23fc6aa882cd401d873519580c5b52ff, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.201262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.206805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 804/1603] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:02 2022] PUT /identity/v3/projects/f3d32df9c64c414d9c93be35820927c9/users/eb6ed6f6c22945e8944ec2414bc2de87/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.208251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.234766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 800/1604] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:02 2022] PUT /identity/v3/projects/0b79dbf6a6ff44b2852078fbc048d413/users/23fc6aa882cd401d873519580c5b52ff/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.234766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.287586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179073-217e-4a46-9f4a-7042e16d3b07 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.333511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2006d-7570-41f3-853b-20dd23120bc1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.340944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.342614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.343062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.343574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] PATH_INFO: `/v3/projects/921ab4465c1b48758a03fe0411940778/users/513cbdb59f2b4466963279513c7f8b06/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.354057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.354959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.355269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.355535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] PATH_INFO: `/v3/projects/d7e9858d04f9453f8aad1d2346a82b22/users/6f0ea24fc72542fcaa0c504dcc73eace/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.361755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] RBAC: Authorizing `identity:create_grant(project_id=921ab4465c1b48758a03fe0411940778, user_id=513cbdb59f2b4466963279513c7f8b06, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.368458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.377914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e9858d04f9453f8aad1d2346a82b22, user_id=6f0ea24fc72542fcaa0c504dcc73eace, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.387273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 801/1605] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:02 2022] PUT /identity/v3/projects/921ab4465c1b48758a03fe0411940778/users/513cbdb59f2b4466963279513c7f8b06/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.388521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.390441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b04aee-84bf-4c7d-a823-eb18604f6c39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.413027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 805/1606] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:02 2022] PUT /identity/v3/projects/d7e9858d04f9453f8aad1d2346a82b22/users/6f0ea24fc72542fcaa0c504dcc73eace/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.415932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd05e-62d6-44f2-869b-99d7cb78f128 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.416218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd05e-62d6-44f2-869b-99d7cb78f128 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.416500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debd05e-62d6-44f2-869b-99d7cb78f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.461871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5debd05e-62d6-44f2-869b-99d7cb78f128 None None] MFA Rules not processed for user `8124939d5de3486989a75ba918c3e71a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.463635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5debd05e-62d6-44f2-869b-99d7cb78f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:02.475461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:02.501916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f7c0-3b1a-4ede-98d4-17c331f9875e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.507211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:02.594187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 806/1607] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:02.595874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600481cd-a0bd-4482-bad2-8e0fdce29ee5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.596186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600481cd-a0bd-4482-bad2-8e0fdce29ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.596527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600481cd-a0bd-4482-bad2-8e0fdce29ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.622679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-600481cd-a0bd-4482-bad2-8e0fdce29ee5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.629163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-600481cd-a0bd-4482-bad2-8e0fdce29ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:02.650905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.652443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.652726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.653000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] PATH_INFO: `/v3/projects/40936219a8dc4c01924fcbe13afee48d/users/190f2f5fc7f3481a8f1d5772f605d924/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.688991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40936219a8dc4c01924fcbe13afee48d, user_id=190f2f5fc7f3481a8f1d5772f605d924, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.691807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64e634-7e73-4e10-9dc4-751fb2c15a0c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.713613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 802/1608] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:02 2022] PUT /identity/v3/projects/40936219a8dc4c01924fcbe13afee48d/users/190f2f5fc7f3481a8f1d5772f605d924/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:02.715122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.715434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.716415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.724310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 807/1609] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:02.726160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118be53-836b-4752-b988-78538569b198 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.726508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118be53-836b-4752-b988-78538569b198 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.726849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118be53-836b-4752-b988-78538569b198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.749274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0118be53-836b-4752-b988-78538569b198 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.750643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0118be53-836b-4752-b988-78538569b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:02.772089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] MFA Rules not processed for user `23fc6aa882cd401d873519580c5b52ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.774672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:02.844239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 808/1610] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:02.846028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.846419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.846796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.862055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 803/1611] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:02.864163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.877311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675f09f-97f5-472a-9614-62c4f3eaee92 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:02.884585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:02.885372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.885657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.886062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.886765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:02.888756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83518f0-9cdd-4ca3-a3b6-1db5de9af0c8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:02.889289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] MFA Rules not processed for user `6f0ea24fc72542fcaa0c504dcc73eace`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.893486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:02.928586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 804/1612] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:02.930622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:02.931254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:02.931254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:02.944035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:02.975614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] MFA Rules not processed for user `190f2f5fc7f3481a8f1d5772f605d924`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:02.981098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 809/1613] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:02.981098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:02.981361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:03.057369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:03.083007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5426-10b2-4482-a2c3-9caa200e5177 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.087933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 805/1614] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:03.088952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.157836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.158474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e170-0d44-4e0d-84c7-0863a3795374 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.158618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.158824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.159105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.162386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.163028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.163359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.164486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.239608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.242568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c968c3-ae61-455d-8c8f-f1284f7a778f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.245280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.247826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075f4d4-a66f-4b48-b18d-a49086f0ebf1 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.256544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 810/1615] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:03.258528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b3c00-f797-481b-b541-2f7a03b6bb94 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.258912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b3c00-f797-481b-b541-2f7a03b6bb94 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.259330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b3c00-f797-481b-b541-2f7a03b6bb94 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.261223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 811/1616] 10.222.0.39 () {54 vars in 961 bytes} [Thu May 26 03:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:03.261882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b3c00-f797-481b-b541-2f7a03b6bb94 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.262208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 806/1617] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:03.263645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22ccf2-5ca5-4713-84f7-059684ad3e53 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.263935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22ccf2-5ca5-4713-84f7-059684ad3e53 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.264214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22ccf2-5ca5-4713-84f7-059684ad3e53 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.275275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 807/1618] 10.222.0.39 () {54 vars in 961 bytes} [Thu May 26 03:19:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:03.276332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22ccf2-5ca5-4713-84f7-059684ad3e53 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.283845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b3c00-f797-481b-b541-2f7a03b6bb94 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.290908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.291858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.292353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.292718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.293472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.294232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22ccf2-5ca5-4713-84f7-059684ad3e53 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.296064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6d2f-4460-458c-9b23-42bc85465d0c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.299968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.300871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.301272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.301626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.303013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.306513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b53615-b80c-4fdf-a3b4-0dac390f0895 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.329301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 812/1619] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:03 2022] POST /identity/v3/users => generated 292 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:03.331415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.331834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.332080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.349111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 808/1620] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:03 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:03.351283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.351796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.352219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.353056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:03.354327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:03.373050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:03.374676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:03.382377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:03.445932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 813/1621] 10.222.0.39 () {58 vars in 1058 bytes} [Thu May 26 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:03.447329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.464499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 809/1622] 10.222.0.39 () {58 vars in 1058 bytes} [Thu May 26 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:03.465461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.477354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:03.528542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf10ce-8ed1-406e-bdbc-fc3a677a6371 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.534763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.535804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.536296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.536763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.537766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.540201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.548121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fa62-f045-4637-a3ad-88d615f75398 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.553303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.557535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.557535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.557535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.557535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.558329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 814/1623] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:03.558448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.559554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.578498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609adea-c7e4-41fe-9175-9ad2ac1bbe0b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.585345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 810/1624] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:03 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:03.586685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.596870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.597956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.601633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.601957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.604334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac1125-fe3c-48c9-bd3a-7d372d18ee44 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.612893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.615893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.617247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.618203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.688317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.690038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.693005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.696151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.702726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 811/1625] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:03.703938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.716628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7476-27e6-4ae7-8c0c-9c08152c4442 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.721670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.722876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.723281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.723659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] PATH_INFO: `/v3/projects/7d4a099240bd4a0fbd440a1a2873860f/users/64023aee329447b289e12cceba4f2a04/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.724733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 815/1626] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:03.725681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.741508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4a099240bd4a0fbd440a1a2873860f, user_id=64023aee329447b289e12cceba4f2a04, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.744939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.753399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7202a-0d05-4b80-a6e4-114442921f4a service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.767857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.769187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 812/1627] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:03 2022] PUT /identity/v3/projects/7d4a099240bd4a0fbd440a1a2873860f/users/64023aee329447b289e12cceba4f2a04/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:03.769616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.769996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.770368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.772974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.774339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.777059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.796232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 816/1628] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:03.800328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:03.826279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c473c-6e5b-474a-88a2-6efc5dcef509 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.912804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2817-4e87-4a04-b9ff-1157517f9b69 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:03.916713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:03.923608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.924624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.925806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.927379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.928925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:03.932348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:03.950496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:03.952526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:03.952851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:03.954146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] PATH_INFO: `/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8/users/40091721714c48d8bec10586cf27aaeb/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:03.958896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 813/1629] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:03.962125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.007545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2a4fd-51e6-40eb-b9e1-332eb8dbf119 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.018774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5eb2e921dfb4ef59f2cd0c687fb68c8, user_id=40091721714c48d8bec10586cf27aaeb, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.027769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.030139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.032516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.035994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.036495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] PATH_INFO: `/v3/projects/7d4a099240bd4a0fbd440a1a2873860f/users/64023aee329447b289e12cceba4f2a04/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.044111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4a099240bd4a0fbd440a1a2873860f, user_id=64023aee329447b289e12cceba4f2a04, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.050183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b6cc8-3fdf-4550-81f8-fe3960b94c20 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.062568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:04.068941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 817/1630] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:03 2022] PUT /identity/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8/users/40091721714c48d8bec10586cf27aaeb/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:04.072162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.096668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 814/1631] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:03 2022] PUT /identity/v3/projects/7d4a099240bd4a0fbd440a1a2873860f/users/64023aee329447b289e12cceba4f2a04/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:04.108643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b7e1-e9a9-4d7a-83e2-b1d538813526 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.109150 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b7e1-e9a9-4d7a-83e2-b1d538813526 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.109542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b7e1-e9a9-4d7a-83e2-b1d538813526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.162208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a093b7e1-e9a9-4d7a-83e2-b1d538813526 None None] MFA Rules not processed for user `64023aee329447b289e12cceba4f2a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:04.166264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a093b7e1-e9a9-4d7a-83e2-b1d538813526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:04.211829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0bfbb-4c9d-4691-9d2d-5d283b8230db admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.328573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 815/1632] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:04.331260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b28952-671e-483e-85e9-245875c08daf None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.331581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b28952-671e-483e-85e9-245875c08daf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.331861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b28952-671e-483e-85e9-245875c08daf None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.342129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 816/1633] 10.222.0.39 () {54 vars in 912 bytes} [Thu May 26 03:19:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:04.343584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47ff9-4312-471b-b33c-5e259e480772 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.343884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47ff9-4312-471b-b33c-5e259e480772 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.344685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47ff9-4312-471b-b33c-5e259e480772 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.346700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.363666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 817/1634] 10.222.0.39 () {54 vars in 912 bytes} [Thu May 26 03:19:04 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:04.365000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.365292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.365570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.367950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.368575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.369088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.370143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.372226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.388426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 818/1635] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:04.389770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.398943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:04.403539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:04.480426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2648aa4-a5d9-4240-9e78-2e82709edd55 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.493400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.494128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.494456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.494785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.518642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.522049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1816-11f6-4bbe-bd2b-92306aeb601e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.545063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 819/1636] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:04.547488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a0e9-dacc-418b-8280-cb047c083002 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.547984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a0e9-dacc-418b-8280-cb047c083002 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.548362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a0e9-dacc-418b-8280-cb047c083002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.548876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 818/1637] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:04.557723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.572205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61060bc-c4d7-43bf-becf-73f84b2c1b7f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.580907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.582013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.582302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.582577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] PATH_INFO: `/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8/users/40091721714c48d8bec10586cf27aaeb/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.585296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5eb2e921dfb4ef59f2cd0c687fb68c8, user_id=40091721714c48d8bec10586cf27aaeb, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.589126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.590027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-44c7a0e9-dacc-418b-8280-cb047c083002 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:04.616203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-44c7a0e9-dacc-418b-8280-cb047c083002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:04.635699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 819/1638] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:04 2022] PUT /identity/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8/users/40091721714c48d8bec10586cf27aaeb/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:04.636793 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.661169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:04.779845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a41b-c31b-4bbd-a821-beccab3b3bbd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.786155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.787126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.787813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.788105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.793013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 820/1639] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:04.795153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac3a-2259-40b9-863c-db2fddc5e38b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.796215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac3a-2259-40b9-863c-db2fddc5e38b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.797062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dac3a-2259-40b9-863c-db2fddc5e38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.813306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.817115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.829919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:04.844228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 820/1640] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:04.845344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:04.856282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b70dac3a-2259-40b9-863c-db2fddc5e38b None None] MFA Rules not processed for user `40091721714c48d8bec10586cf27aaeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:04.857377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9071-8079-48d4-9fba-39b894dab6b9 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:04.858644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b70dac3a-2259-40b9-863c-db2fddc5e38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:04.862001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:04.863112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:04.863553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:04.863957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:04.879008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:04.881294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:04.898916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 821/1641] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:05.071969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 821/1642] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:05.453837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:05.468796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dd87d-0859-4f71-98ea-3107a0b2f3cf service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:05.475676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:05.476894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:05.477181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:05.477465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:05.492782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:05.494784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:05.526322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 822/1643] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:05.861265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:05.861774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:05.861868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:05.868005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:05.882885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b4eed-77b4-4394-afe8-349ce51ce03c service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:05.891772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:05.892493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:05.892799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:05.893658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:05.937701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] MFA Rules not processed for user `eb6ed6f6c22945e8944ec2414bc2de87`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:05.941645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:06.044025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:06.051089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a51d55-55e6-4b24-9ee8-9672a6751840 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:06.083404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 823/1644] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:19:06.115713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 822/1645] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:06.119972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:06.120542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:06.120926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:06.144808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] MFA Rules not processed for user `eb6ed6f6c22945e8944ec2414bc2de87`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:06.146446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:06.174596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 824/1646] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:06.230533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:06.232283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:06.232283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089072cc-6e7c-4195-b16d-afa6437c4418 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:06.245570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:06.250721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:06.251355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:06.251792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:06.268379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:06.273739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:06.273949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a299d5-fd18-4246-8d2e-bbcfaba89310 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:06.299688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 825/1647] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:19:06.359858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2f099-c2e3-4701-a6f8-4b3b37482ae1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:06.368167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:06.368167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:06.368167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:06.368167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] PATH_INFO: `/v3/users/6f0ea24fc72542fcaa0c504dcc73eace` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:06.369779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0ea24fc72542fcaa0c504dcc73eace)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:06.370211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: WARNING py.warnings [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:06.370211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: warnings.warn(msg) May 26 03:19:06.370211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: May 26 03:19:06.371564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:06.503700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0ea24fc72542fcaa0c504dcc73eace'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:06.504390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0ea24fc72542fcaa0c504dcc73eace'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:06.528499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0ea24fc72542fcaa0c504dcc73eace'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:06.531503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0ea24fc72542fcaa0c504dcc73eace'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:06.584067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0ea24fc72542fcaa0c504dcc73eace'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:06.618499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 823/1648] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:06 2022] DELETE /identity/v3/users/6f0ea24fc72542fcaa0c504dcc73eace => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:06.623640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d215-da0e-41b2-a9f8-714672c66e29 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:06.624192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d215-da0e-41b2-a9f8-714672c66e29 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:06.624380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22d215-da0e-41b2-a9f8-714672c66e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:06.690454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9a22d215-da0e-41b2-a9f8-714672c66e29 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:06.693705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9a22d215-da0e-41b2-a9f8-714672c66e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:06.893118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:06.894732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:06.920330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 826/1649] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:06.924508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:06.925088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:06.925531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:06.935121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 827/1650] 10.222.0.39 () {54 vars in 912 bytes} [Thu May 26 03:19:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:06.937030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:06.939728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:07.030582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4429ef2-ff94-42d4-bb8e-fc8a160e5a46 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:07.036024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:07.037924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.038206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.039122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] PATH_INFO: `/v3/users/8124939d5de3486989a75ba918c3e71a` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.040473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8124939d5de3486989a75ba918c3e71a)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:07.042844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:07.077935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b0175-cb2d-4c51-8bf7-0208a7b70eae None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:07.082655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:07.083471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.083857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.084201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.101112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:07.106395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:07.107950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d544d-7db8-4d05-b468-b926a742fdf2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:07.115361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:07.124098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8124939d5de3486989a75ba918c3e71a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:07.125839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8124939d5de3486989a75ba918c3e71a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:07.137563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8124939d5de3486989a75ba918c3e71a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:07.139053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8124939d5de3486989a75ba918c3e71a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:07.144975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 828/1651] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:07.149154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.149466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.149757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.168407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-139f7abf-c20d-4cf7-bb13-c4cf8a29b1c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8124939d5de3486989a75ba918c3e71a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:07.205850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 824/1652] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:06 2022] DELETE /identity/v3/users/8124939d5de3486989a75ba918c3e71a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:07.212132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.213000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.213505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.300722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:07.303604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:07.313839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:07.315856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:07.531661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 825/1653] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:07.535344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 829/1654] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:19:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:07.582916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:07.585713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:07.613650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e4f58-0a81-4ff3-9792-e2c32a5e2379 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:07.627259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:07.627965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.628254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.630873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.654409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:07.659584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:07.667060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:07.677942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7e7fc-68b9-42c8-81c8-159a49c68eb6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:07.688215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:07.689294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:07.689419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 826/1655] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:07.689960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:07.690286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:07.721081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:07.731628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:07.755724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 830/1656] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:08.307374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:08.308830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:08.335956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20805716-5c34-444a-a09c-5c59ee103d54 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:08.347669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:08.349018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:08.349440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:08.350611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] PATH_INFO: `/v3/projects/d7e9858d04f9453f8aad1d2346a82b22` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:08.355548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e9858d04f9453f8aad1d2346a82b22)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:08.358401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:08.534558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:08.536466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:08.537243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e9858d04f9453f8aad1d2346a82b22', 'request_id': 'req-000d720b-0d5b-4111-9a26-997856d8932e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.562737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e9858d04f9453f8aad1d2346a82b22', 'request_id': 'req-000d720b-0d5b-4111-9a26-997856d8932e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.568685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92d39f-9a8e-4b09-929d-295ca8ff1d9d service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:08.573742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e9858d04f9453f8aad1d2346a82b22', 'request_id': 'req-000d720b-0d5b-4111-9a26-997856d8932e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.590457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] The token cache is being invalidate because project d7e9858d04f9453f8aad1d2346a82b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:08.591253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.592466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-000d720b-0d5b-4111-9a26-997856d8932e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.594555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 827/1657] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:08 2022] DELETE /identity/v3/projects/d7e9858d04f9453f8aad1d2346a82b22 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:08.599217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:08.599670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:08.600165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:08.673664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:08.674995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:08.700336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:08.701912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:08.702337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:08.703204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] PATH_INFO: `/v3/projects/22ca67d4b5de455cb8dc976e05776cf3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:08.705051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ca67d4b5de455cb8dc976e05776cf3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:08.706408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: WARNING py.warnings [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 03:19:08.706408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: warnings.warn(msg) May 26 03:19:08.706408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: May 26 03:19:08.708228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:08.733357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:08.817293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 828/1658] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:08.817293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:08.817293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:08.841055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ca67d4b5de455cb8dc976e05776cf3', 'request_id': 'req-48a4f49f-6117-4d0b-a7e7-12f72caf419b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.855084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ca67d4b5de455cb8dc976e05776cf3', 'request_id': 'req-48a4f49f-6117-4d0b-a7e7-12f72caf419b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.906552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ca67d4b5de455cb8dc976e05776cf3', 'request_id': 'req-48a4f49f-6117-4d0b-a7e7-12f72caf419b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.938160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] The token cache is being invalidate because project 22ca67d4b5de455cb8dc976e05776cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:08.938658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.939405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-48a4f49f-6117-4d0b-a7e7-12f72caf419b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:08.951913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 831/1659] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:08 2022] DELETE /identity/v3/projects/22ca67d4b5de455cb8dc976e05776cf3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:08.964751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:08.967615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:08.968278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:08.969633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ba564-18bb-4c59-b2d4-8b8046bbb1d7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.030019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:09.033053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:09.059296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:09.128716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.129422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.129710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.129988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.136764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:09.164931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.166584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.191912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 829/1660] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:09.192752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:09.195520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:09.245073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 832/1661] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:09.248825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:09.274867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78901b3d-6e14-427f-b40d-1466452f1196 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.292536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.293142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.293442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.293703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.294448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:09.294758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.296485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.346628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 833/1662] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:19:09.348117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:09.380869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f5c7-e9d7-4191-9681-368c2a3b0425 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.393679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.394440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.394745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.395014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.395822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.397864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.408009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914af261-84ef-45c4-a2e1-462684871e23 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.421960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.423201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.423747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.424665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] PATH_INFO: `/v3/users/64023aee329447b289e12cceba4f2a04` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.426595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64023aee329447b289e12cceba4f2a04)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.429591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.439947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:09.477783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 834/1663] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:09 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:09.478358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:09.534931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64023aee329447b289e12cceba4f2a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:09.540637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64023aee329447b289e12cceba4f2a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:09.555287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64023aee329447b289e12cceba4f2a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:09.556663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64023aee329447b289e12cceba4f2a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:09.598335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64023aee329447b289e12cceba4f2a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:09.610111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77235068-9399-4fb9-9987-a54592b61146 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.615919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.617316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.617855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.618143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.619205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.628012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.638337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 830/1664] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:09 2022] DELETE /identity/v3/users/64023aee329447b289e12cceba4f2a04 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:09.721997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:09.724057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:09.731629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 835/1665] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:09 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:09.938274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64276fe-05c6-4430-bb42-eae2abb42c9a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:09.954530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:09.956754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:09.957072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:09.957355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:09.958263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:09.966469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:09.995060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:10.005468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 831/1666] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:10.013086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443543b-de44-4e58-baf2-6be7920c5d40 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:10.021470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:10.022173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:10.022462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:10.023468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:10.029828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:10.031851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2bae2-2661-463e-ae77-60dd36b91b9f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:10.064001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:10.071198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 836/1667] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:10.086970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6d855-0f0c-4fe6-8314-dcb6cd8ec27f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:10.104035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:10.107713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:10.109261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:10.113734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] PATH_INFO: `/v3/projects/73dd0d77849e47b5847e97b655f442aa/users/575bca22c34344f28e8f546df0b98038/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:10.161890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dd0d77849e47b5847e97b655f442aa, user_id=575bca22c34344f28e8f546df0b98038, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:10.166094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:10.215511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 832/1668] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:10 2022] PUT /identity/v3/projects/73dd0d77849e47b5847e97b655f442aa/users/575bca22c34344f28e8f546df0b98038/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:10.227262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:10.227564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:10.227847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:10.296843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] MFA Rules not processed for user `575bca22c34344f28e8f546df0b98038`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:10.298094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:10.377280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:10.461467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:10.462746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 837/1669] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:10.640169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:10.676549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ce157-4a99-4a64-a3df-d44396127db8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:10.688559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:10.689340 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:10.690300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:10.690656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:10.691450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:10.693573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:10.774281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 833/1670] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:10 2022] POST /identity/v3/projects => generated 369 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:10.777831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:10.794906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5bfce-7e34-4f96-8b37-e174284ce08c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:10.996396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:10.998390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:11.073136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:11.074243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.074607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.079705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.080569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:11.089942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:11.093277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff1b72-9f8e-434e-99d6-479ba800a3b6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:11.105633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:11.107967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.108919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.109780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] PATH_INFO: `/v3/projects/7d4a099240bd4a0fbd440a1a2873860f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.115618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4a099240bd4a0fbd440a1a2873860f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:11.118437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:11.187973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 838/1671] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:10 2022] POST /identity/v3/users => generated 291 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:11.189361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:11.214512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8910-f994-42df-b89a-82d8feeca062 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:11.246693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:11.247668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.254880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.255412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.256326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:11.259641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:11.297109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 839/1672] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:11.297775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4a099240bd4a0fbd440a1a2873860f', 'request_id': 'req-84a0b884-bb54-4c30-993d-42afde6b9ca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:11.305620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:11.323043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4a099240bd4a0fbd440a1a2873860f', 'request_id': 'req-84a0b884-bb54-4c30-993d-42afde6b9ca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:11.325773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b10d47-85c0-44ae-89fb-b3d69194f1c8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:11.348443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4a099240bd4a0fbd440a1a2873860f', 'request_id': 'req-84a0b884-bb54-4c30-993d-42afde6b9ca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:11.407505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] The token cache is being invalidate because project 7d4a099240bd4a0fbd440a1a2873860f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:11.407804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:11.409938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-84a0b884-bb54-4c30-993d-42afde6b9ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:11.412608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 834/1673] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:10 2022] DELETE /identity/v3/projects/7d4a099240bd4a0fbd440a1a2873860f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:11.420068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dcc-6697-4272-b5bb-5db5b60a7596 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.420352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dcc-6697-4272-b5bb-5db5b60a7596 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.420647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f1dcc-6697-4272-b5bb-5db5b60a7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.468768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab6f1dcc-6697-4272-b5bb-5db5b60a7596 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:11.472883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab6f1dcc-6697-4272-b5bb-5db5b60a7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:11.496417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:11.500133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.500669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.500969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] PATH_INFO: `/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82/users/27480e60f0914a828abeea285741783d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.547227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f2eac1de3e46d3b9966e1489fa4d82, user_id=27480e60f0914a828abeea285741783d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:11.555996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:11.605911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 840/1674] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:11 2022] PUT /identity/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82/users/27480e60f0914a828abeea285741783d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:11.613897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:11.621559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:11.660592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 835/1675] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:11.670537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.673064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.673522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.739825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:11.740942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:11.765934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:11.804539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:11.880165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff42c-c5ca-4e49-87e8-95ea23255421 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:11.907989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:11.908687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:11.908969 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:11.909242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:11.909978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:11.911620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:11.946682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 836/1676] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:11.950856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:11.955664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 841/1677] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:11.956543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:11.981397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0369116-718e-49a2-acb3-3c26c1e8ce8d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:11.982109 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a152e-053d-4079-a9ac-545180e6931c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:12.001308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:12.004740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.006169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.006481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] PATH_INFO: `/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82/users/27480e60f0914a828abeea285741783d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.009251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:12.009785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.010075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.010372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.011104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:12.011569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:12.013352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:12.017112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f2eac1de3e46d3b9966e1489fa4d82, user_id=27480e60f0914a828abeea285741783d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:12.030254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf948b-4594-4387-969e-776d70c4b0c4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:12.061358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 837/1678] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:12.066368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:12.085832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 842/1679] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:11 2022] PUT /identity/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82/users/27480e60f0914a828abeea285741783d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:12.097321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f748b6-0b21-4d84-832a-be779eccfda4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:12.102368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.102683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.111628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.330191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:12.339156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.342305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.343367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.352844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:12.365156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:12.487352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 838/1680] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:12 2022] POST /identity/v3/projects => generated 377 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:12.488354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:12.677415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] MFA Rules not processed for user `27480e60f0914a828abeea285741783d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:12.681610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:12.743082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5615cf-d598-4a28-a0d8-067df1ee4c5a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:12.761904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:12.771100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.773419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.775829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.779796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:12.791663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:12.895455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 839/1681] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:12 2022] POST /identity/v3/users => generated 289 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:12.897580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:12.905141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:12.950329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebf9e5-60f7-473e-b642-5ad3afe7f6b3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:12.966593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:12.967854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:12.968477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:12.970114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] PATH_INFO: `/v3/users/190f2f5fc7f3481a8f1d5772f605d924` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:12.971878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190f2f5fc7f3481a8f1d5772f605d924)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:12.977788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:13.008596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:13.021738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 843/1682] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 926 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:13.024820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:13.076816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954976e5-dfb1-4ba7-9e97-22acf34f4f28 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:13.080823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190f2f5fc7f3481a8f1d5772f605d924'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.081754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190f2f5fc7f3481a8f1d5772f605d924'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.102428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:13.103122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.103477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.104064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.106041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:13.111128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:13.118976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190f2f5fc7f3481a8f1d5772f605d924'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.120767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190f2f5fc7f3481a8f1d5772f605d924'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.161674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 844/1683] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:13.172507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:13.175278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.177587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190f2f5fc7f3481a8f1d5772f605d924'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.217646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 840/1684] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:12 2022] DELETE /identity/v3/users/190f2f5fc7f3481a8f1d5772f605d924 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:13.221265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:13.225295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.350345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b5af-5d4d-44af-9461-d5560b83ed58 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:13.400942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a2e1d-8eeb-414d-97fa-a8d19428d708 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:13.412844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:13.413573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.413859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.414135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.421848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:13.423574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:13.462849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 841/1685] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:13.464002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:13.472105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.489262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:13.489928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.490206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.490476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.496654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.499437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb74e-fb6f-4ac8-874e-48670510bc2e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:13.517113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:13.519563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.521127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.523362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] PATH_INFO: `/v3/users/23fc6aa882cd401d873519580c5b52ff` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.528458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:13.529404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fc6aa882cd401d873519580c5b52ff)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:13.543284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:13.678445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fc6aa882cd401d873519580c5b52ff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.679165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fc6aa882cd401d873519580c5b52ff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.682231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:13.683727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fb97b-7ca4-4fa6-902e-79f549cc88b4 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:13.700625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fc6aa882cd401d873519580c5b52ff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.700625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fc6aa882cd401d873519580c5b52ff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.702993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 845/1686] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:13.704186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ad1e-56af-4814-834a-00824e24f039 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.704475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ad1e-56af-4814-834a-00824e24f039 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.704753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ad1e-56af-4814-834a-00824e24f039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.721597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fc6aa882cd401d873519580c5b52ff'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:13.724510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:13.749025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2287ad1e-56af-4814-834a-00824e24f039 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:13.754307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2287ad1e-56af-4814-834a-00824e24f039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.754589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 842/1687] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:13 2022] DELETE /identity/v3/users/23fc6aa882cd401d873519580c5b52ff => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:13.758193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:13.759557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:13.920580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb9c87-6613-4f7a-9220-8e684fe8075d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:13.927052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:13.929157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.929518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.929805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] PATH_INFO: `/v3/projects/315f9bf502054d759f0a2dae89fabe55/users/313f40bacca245f79aa0dc4ba710c605/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.937977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 846/1688] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:13.939392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:13.939692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:13.939958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:13.963201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315f9bf502054d759f0a2dae89fabe55, user_id=313f40bacca245f79aa0dc4ba710c605, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:13.968718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab08dcb-d97b-47bf-bf01-e36b3388b2d1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:14.016726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:14.017953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:14.031725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 843/1689] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:13 2022] PUT /identity/v3/projects/315f9bf502054d759f0a2dae89fabe55/users/313f40bacca245f79aa0dc4ba710c605/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:14.033424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a9d31-e5c0-444c-ad26-e699e422366a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:14.035115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a9d31-e5c0-444c-ad26-e699e422366a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:14.036364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a9d31-e5c0-444c-ad26-e699e422366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:14.082956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9b6a9d31-e5c0-444c-ad26-e699e422366a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:14.086115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9b6a9d31-e5c0-444c-ad26-e699e422366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:14.111138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 847/1690] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:19:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:14.112123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:14.113609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:14.206235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bf08b-5e15-4656-a7d3-016554200b2a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:14.212663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:14.213553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:14.214077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:14.214686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:14.228872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:14.242090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 844/1691] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:14.242090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:14.242278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:14.242500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:14.322017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] MFA Rules not processed for user `313f40bacca245f79aa0dc4ba710c605`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:14.327113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:14.484855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:14.489794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:14.511151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 848/1692] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:14.534830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:14.574685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404b9e7-863b-4020-b325-11aed1c46600 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:14.583967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:14.586443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:14.586873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:14.587161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:14.606768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:14.610184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:14.641806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 849/1693] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:14.642846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:14.644530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:14.666644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 845/1694] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:14.670845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:14.688548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:14.703224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be130b1-90ec-4619-80cc-bac4d6d7d0ec None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:14.720944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:14.721813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:14.722194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:14.722566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:14.723809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:14.731956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:14.797136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 846/1695] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:14 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:14.801137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:14.850202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136068-1412-4f49-96b3-0a9ae91d4567 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:15.074431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f9491-1497-40e4-b9e6-7a2b9cd64ed0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:15.074897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:15.075603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.075890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.078370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.091952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:15.094797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.095260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.096813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.098181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:15.103396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:15.154123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 847/1696] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:14 2022] POST /identity/v3/users => generated 294 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:15.157395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:15.174977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:15.185627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010db4b-3fe2-4a45-a233-ccaaa4033404 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:15.208503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:15.211761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.214537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.214819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.215635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:15.217277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:15.238643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:15.240797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:15.269812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 848/1697] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:15.270679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:15.275151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 850/1698] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:19:15.299318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5749cc-162b-4bd8-96ad-17f71b292994 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:15.312130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:15.313089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.313321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.313609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] PATH_INFO: `/v3/projects/dc60a823692049178d3fc60433bee1cb/users/880bf10d56634f31b37fdf613b73da3e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.370482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc60a823692049178d3fc60433bee1cb, user_id=880bf10d56634f31b37fdf613b73da3e, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:15.385775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6abcc-8530-46ff-a5a2-0d5e4a6eae4b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:15.445916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 849/1699] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:15 2022] PUT /identity/v3/projects/dc60a823692049178d3fc60433bee1cb/users/880bf10d56634f31b37fdf613b73da3e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:15.446624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:15.489103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49779b-3ba9-48cf-95f6-a0881ae8c7a4 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:15.577163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.577479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.577791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.673449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] MFA Rules not processed for user `513cbdb59f2b4466963279513c7f8b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:15.675701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:15.781019 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 850/1700] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:15.802286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:15.809247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:15.842622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:15.843350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:15.843741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:15.843910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:15.844653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:15.851687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:15.900709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 851/1701] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:15.901171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:15.905146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:15.961206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403635-7aa4-4ca8-ace2-acacefacf32e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:16.004427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:16.009456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.009755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.010043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] PATH_INFO: `/v3/projects/dc60a823692049178d3fc60433bee1cb/users/880bf10d56634f31b37fdf613b73da3e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.062291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc60a823692049178d3fc60433bee1cb, user_id=880bf10d56634f31b37fdf613b73da3e, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:16.087367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859df4d2-b476-4583-a10a-9265ce8cf145 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:16.166250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 852/1702] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:15 2022] PUT /identity/v3/projects/dc60a823692049178d3fc60433bee1cb/users/880bf10d56634f31b37fdf613b73da3e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:16.179147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91af35-5d35-49b7-8319-30f815b4b0c1 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.179884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91af35-5d35-49b7-8319-30f815b4b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.179884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91af35-5d35-49b7-8319-30f815b4b0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.208850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab018f2-43f9-4f8c-b933-cd6db4e4c014 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:16.262168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3e91af35-5d35-49b7-8319-30f815b4b0c1 None None] MFA Rules not processed for user `880bf10d56634f31b37fdf613b73da3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:16.264697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3e91af35-5d35-49b7-8319-30f815b4b0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:16.421729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:16.433439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.433884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.434156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.486720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 853/1703] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:16.491797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.492379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.494946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.558773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] MFA Rules not processed for user `3cf837e2ab5b4e58844a1023cf5f9237`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:16.559871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:16.648351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:16.652290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:16.655374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:16.722098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 851/1704] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 920 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:16.723088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:16.736753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:16.758137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 854/1705] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:16.759895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:16.767988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:16.948344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b70c-7adb-4126-a1ae-07a989ea3479 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:16.949023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec31ecc-a378-4668-bf13-7332304da94e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:16.954581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:16.955704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.956196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.956865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] PATH_INFO: `/v3/projects/40936219a8dc4c01924fcbe13afee48d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.957217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:16.958003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:16.958373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40936219a8dc4c01924fcbe13afee48d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:16.959079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:16.959383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:16.962935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:17.014500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:17.088731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40936219a8dc4c01924fcbe13afee48d', 'request_id': 'req-b4e6195c-08e1-49c8-a819-a1c6e479a831'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.097009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40936219a8dc4c01924fcbe13afee48d', 'request_id': 'req-b4e6195c-08e1-49c8-a819-a1c6e479a831'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.108389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40936219a8dc4c01924fcbe13afee48d', 'request_id': 'req-b4e6195c-08e1-49c8-a819-a1c6e479a831'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.159816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] The token cache is being invalidate because project 40936219a8dc4c01924fcbe13afee48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:17.160495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.161737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.164627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 855/1706] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:16 2022] DELETE /identity/v3/projects/40936219a8dc4c01924fcbe13afee48d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:17.166907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:17.168877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:17.227573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:17.229114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:17.281298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 852/1707] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:17.282793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:17.292783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:17.374983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:17.380953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1e1f0-a3de-4685-b68c-cd1cc62490d3 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:17.497020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6195c-08e1-49c8-a819-a1c6e479a831 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:17.534636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:17.541919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:17.542274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:17.542549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:17.561361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:17.643590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:17.644538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:17.644825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:17.645107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] PATH_INFO: `/v3/users/3d0e4c436d5545b1bc5727186db9ba48` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:17.660105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0e4c436d5545b1bc5727186db9ba48)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:17.663696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:17.852788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0e4c436d5545b1bc5727186db9ba48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.854838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0e4c436d5545b1bc5727186db9ba48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.887181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:17.889058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:17.890526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0e4c436d5545b1bc5727186db9ba48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.894196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0e4c436d5545b1bc5727186db9ba48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.932875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 856/1708] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:17.933797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:17.936201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:17.987825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0e4c436d5545b1bc5727186db9ba48'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:17.996064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff24da4-04f6-423f-8c1c-7fa9def887ad service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:18.019936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:18.021222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:18.021637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:18.023352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] PATH_INFO: `/v3/users/40091721714c48d8bec10586cf27aaeb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:18.026379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40091721714c48d8bec10586cf27aaeb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:18.034399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:18.080570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 853/1709] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:17 2022] DELETE /identity/v3/users/3d0e4c436d5545b1bc5727186db9ba48 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:18.082888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:18.101998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:18.157130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40091721714c48d8bec10586cf27aaeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.167732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40091721714c48d8bec10586cf27aaeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.167732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40091721714c48d8bec10586cf27aaeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.179024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40091721714c48d8bec10586cf27aaeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.180312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fb93f1e4-3c7f-44e0-a19c-c7a7c443dab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40091721714c48d8bec10586cf27aaeb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.202987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 857/1710] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:17 2022] DELETE /identity/v3/users/40091721714c48d8bec10586cf27aaeb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:18.205413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:18.209025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:18.209386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:18.275364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:18.279840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:18.307611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:18.396667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00f79-3b1e-4e58-8650-5907e268d070 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:18.486256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 858/1711] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:18.490041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:18.494249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:18.523276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822f5dc-3e43-4dae-b68e-9d42a5e2fad4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:18.536234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:18.537428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:18.538536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:18.539043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] PATH_INFO: `/v3/projects/0b79dbf6a6ff44b2852078fbc048d413` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:18.542826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b79dbf6a6ff44b2852078fbc048d413)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:18.546490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:18.557651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:18.561353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:18.561353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:18.563016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] PATH_INFO: `/v3/users/575bca22c34344f28e8f546df0b98038` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:18.566100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=575bca22c34344f28e8f546df0b98038)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:18.567822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:18.571658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:18.700451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b79dbf6a6ff44b2852078fbc048d413', 'request_id': 'req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.707926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575bca22c34344f28e8f546df0b98038'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.709309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575bca22c34344f28e8f546df0b98038'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.721080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b79dbf6a6ff44b2852078fbc048d413', 'request_id': 'req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.732064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575bca22c34344f28e8f546df0b98038'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.735350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575bca22c34344f28e8f546df0b98038'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.757373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b79dbf6a6ff44b2852078fbc048d413', 'request_id': 'req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.763914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575bca22c34344f28e8f546df0b98038'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.797024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] The token cache is being invalidate because project 0b79dbf6a6ff44b2852078fbc048d413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:18.799558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.801826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e4a3ab5f-4ec2-4a95-b17f-a2deb5cd8e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:18.805336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 854/1712] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:18 2022] DELETE /identity/v3/users/575bca22c34344f28e8f546df0b98038 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:18.807667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:18.811425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 859/1713] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:18 2022] DELETE /identity/v3/projects/0b79dbf6a6ff44b2852078fbc048d413 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:18.811425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4ea6d-b2ff-4c8b-b34e-16b71450f2c4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:18.813199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4ea6d-b2ff-4c8b-b34e-16b71450f2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:18.813404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4ea6d-b2ff-4c8b-b34e-16b71450f2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:18.820072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:18.860387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-13c4ea6d-b2ff-4c8b-b34e-16b71450f2c4 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:18.861574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-13c4ea6d-b2ff-4c8b-b34e-16b71450f2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.019139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae832cce-d9eb-4efd-addf-cadf8990bd77 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.030562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 860/1714] 10.222.0.45 () {58 vars in 1015 bytes} [Thu May 26 03:19:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:19.031846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.032137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.032414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.034402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.035060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.035363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.035633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.041570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.089884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:19.090405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:19.091658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.253816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.255201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9b7d7-d2f3-49a3-b480-9bb66626add1 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.297395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 855/1715] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:19.297395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.297395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.297395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.301885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 861/1716] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:19.302836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.305460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.335391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43558a4d-3302-424d-b3d2-97d68488eb03 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.352764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.353411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.353680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.353955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.355542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:19.356768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.384907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.387931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637adbc-a354-4fc9-b9a6-6aeb8bc58be7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.392784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 856/1717] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:19.393653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.416689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 862/1718] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:19.424933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.425272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.425631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.428082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940fc68-7264-4bf5-87e2-aeef7dea2f7d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.446730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.447468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.447747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.448021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.491986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.504530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.506728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:19.508122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:19.532891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 857/1719] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:19.537460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 863/1720] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:19.539066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.562188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d687-3d29-4d35-9b3f-64322f713d42 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.572898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.574367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.574367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.574367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.574947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:19.575276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.576904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.605378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 858/1721] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:19.609962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.628247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca4fc9-f704-48c6-9b9c-e80f9bb18f48 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.646421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.647838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.648184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.648385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.649211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.651149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.702930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 864/1722] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:19 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:19.707110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.736487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366db5-c2b2-4fda-a1d1-be2ea69b6426 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.858165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.858880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.859159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.859455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.860169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.862201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.910518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.918058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 859/1723] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:19 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:19.918254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:19.924001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ce62-2037-4eba-9d95-39761159064b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.936194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:19.937152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:19.938383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:19.938825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:19.939859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:19.941819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:19.983455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:19.998898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd68f-2ecd-43b5-a650-5d2776ebf355 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:19.999500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 865/1724] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:20.019663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:20.021185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:20.021675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:20.022770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:20.023877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:20.025770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cca9de-bb3c-4770-802b-4f54e89a4616 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:20.065098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:20.081423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 860/1725] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:20.083904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5446b86-0467-4696-9dc1-16c10b5417f0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:20.108693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:20.109763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:20.110164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:20.110546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] PATH_INFO: `/v3/projects/892ea92d619d406d9f4e350f008c3fac/users/e5a0348d621a4edf957492c9eea5d731/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:20.142901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892ea92d619d406d9f4e350f008c3fac, user_id=e5a0348d621a4edf957492c9eea5d731, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:20.145914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:20.178678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 866/1726] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:20 2022] PUT /identity/v3/projects/892ea92d619d406d9f4e350f008c3fac/users/e5a0348d621a4edf957492c9eea5d731/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:20.189228 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd93323-71f4-470a-8a1a-14af445595a9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:20.189634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd93323-71f4-470a-8a1a-14af445595a9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:20.190082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd93323-71f4-470a-8a1a-14af445595a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:20.310526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-dbd93323-71f4-470a-8a1a-14af445595a9 None None] MFA Rules not processed for user `e5a0348d621a4edf957492c9eea5d731`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:20.313054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dbd93323-71f4-470a-8a1a-14af445595a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:20.422350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:20.476423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 861/1727] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:20.487695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:20.489920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:20.812474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f104e-a53c-4884-93a4-d9f17a11dd31 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:20.851963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:20.853420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:20.853700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:20.853991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:20.884386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:20.888531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:20.931674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 867/1728] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:21.573877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:21.574359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:21.574672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:21.655706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] MFA Rules not processed for user `513cbdb59f2b4466963279513c7f8b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:21.680214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:21.926295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 862/1729] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:22.157569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:22.159971 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:22.177687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:22.182398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:22.312762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ae000-ca93-4d85-9a68-51747a36aa61 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:22.320734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:22.322501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.322742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.323211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] PATH_INFO: `/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.324191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d7b60-fe5a-4b26-92c8-365fb4a3b25f service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:22.324589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5eb2e921dfb4ef59f2cd0c687fb68c8)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:22.326684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:22.333117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:22.334771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.335387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.336044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] PATH_INFO: `/v3/projects/421d43c3fe944cfdbe747ea49ff7a905` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.338200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421d43c3fe944cfdbe747ea49ff7a905)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:22.340958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:22.423262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5eb2e921dfb4ef59f2cd0c687fb68c8', 'request_id': 'req-d763cfef-2853-4823-bc73-b70fe6a63767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.453180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5eb2e921dfb4ef59f2cd0c687fb68c8', 'request_id': 'req-d763cfef-2853-4823-bc73-b70fe6a63767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.468471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5eb2e921dfb4ef59f2cd0c687fb68c8', 'request_id': 'req-d763cfef-2853-4823-bc73-b70fe6a63767'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.468829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421d43c3fe944cfdbe747ea49ff7a905', 'request_id': 'req-0235ee61-2cd4-44ce-91cb-74f06a60d985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.479846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:22.486126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] The token cache is being invalidate because project d5eb2e921dfb4ef59f2cd0c687fb68c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:22.486347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.486647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421d43c3fe944cfdbe747ea49ff7a905', 'request_id': 'req-0235ee61-2cd4-44ce-91cb-74f06a60d985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.487635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d763cfef-2853-4823-bc73-b70fe6a63767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.493291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 863/1730] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:22 2022] DELETE /identity/v3/projects/d5eb2e921dfb4ef59f2cd0c687fb68c8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:22.500197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947cbc-db35-4feb-97f0-b820602a995e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.500412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947cbc-db35-4feb-97f0-b820602a995e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.500608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1947cbc-db35-4feb-97f0-b820602a995e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.517005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421d43c3fe944cfdbe747ea49ff7a905', 'request_id': 'req-0235ee61-2cd4-44ce-91cb-74f06a60d985'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.549713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] The token cache is being invalidate because project 421d43c3fe944cfdbe747ea49ff7a905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:22.549933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.551332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0235ee61-2cd4-44ce-91cb-74f06a60d985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:22.558618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 868/1731] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:22 2022] DELETE /identity/v3/projects/421d43c3fe944cfdbe747ea49ff7a905 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:22.563291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01b50-6555-4161-8c3b-aa7c25c42d4c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.565679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01b50-6555-4161-8c3b-aa7c25c42d4c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.566794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab01b50-6555-4161-8c3b-aa7c25c42d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.579990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f1947cbc-db35-4feb-97f0-b820602a995e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:22.584298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f1947cbc-db35-4feb-97f0-b820602a995e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:22.629178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dab01b50-6555-4161-8c3b-aa7c25c42d4c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:22.630891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dab01b50-6555-4161-8c3b-aa7c25c42d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:22.659986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:22.788772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 864/1732] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:22.798730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.798951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.799437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.881445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:22.882713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:22.936427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 869/1733] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:22.946188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 865/1734] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:22.950611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:22.956477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.956917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:22.957159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:22.979894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e486fe8-011b-459c-a373-b7ce4c71a55d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:22.997608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:22.998684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:22.999383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.000014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.001224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:23.001884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.003901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.027107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 866/1735] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:19:23.029550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.033332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:23.034605 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:23.036916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:23.054527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 870/1736] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:23.055062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.081895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04111ed0-e587-4c01-a007-4b4c39b3d999 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.086989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.087958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.088248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.089180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.089273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.091445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.144779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 871/1737] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:23 2022] POST /identity/v3/projects => generated 393 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:23.145747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.181942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236691-ba2f-4ebe-9c5d-292394bc0e62 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.290151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc0190-0784-438e-9360-afe7aa2afdea admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.293653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.294390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.295202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.295905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.299509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:23.302657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.304048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.304940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.306030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.308666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:23.309071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.315749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.359466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 872/1738] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:23.361315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.393351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270733-0c4e-462c-9445-cee6c26e6303 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.406311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.409133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.409502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.409788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.413939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.417398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.458902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.462234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.482674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 873/1739] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:23 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:23.485186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.500604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 867/1740] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:23.503697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.507900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2e2ec-1c4a-4285-95a6-692fb42c7627 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.520248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.525924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.525924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.525924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.525924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.528039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffcf3a-25b6-45ee-9a6e-94bf2b492803 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.529289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.541575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.544615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.546369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.547188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.550435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.558640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.578051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:23.599030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 874/1741] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:23 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:23.602669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.608109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 868/1742] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:23.612387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.642999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:23.646173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644df665-e812-41b4-9133-ef0768b819eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.811155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.812129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.812428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.812790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.814939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.822118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.837950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e58da-46d2-44de-84cb-a81df58e69e7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.852302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 869/1743] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:23.853272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.853730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.855011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.855576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.855855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.856561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.864144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:23.869649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.897914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615ba95-f491-4688-a96b-8802e344fbe9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:23.907702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:23.908182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:23.908606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:23.908939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] PATH_INFO: `/v3/projects/73dd0d77849e47b5847e97b655f442aa` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:23.911367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=73dd0d77849e47b5847e97b655f442aa)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:23.916056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:23.948999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 875/1744] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:23 2022] POST /identity/v3/users => generated 290 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:23.950271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:23.974684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e8c9-ec5c-4553-8549-ec1849eefdf4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:24.006244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:24.009626 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.009914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.012079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] PATH_INFO: `/v3/projects/f9390af8e9f645c3b446a1bae563b0ae/users/97fedba50781434e892c22142aba4dab/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.059593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9390af8e9f645c3b446a1bae563b0ae, user_id=97fedba50781434e892c22142aba4dab, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:24.065712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:24.087108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73dd0d77849e47b5847e97b655f442aa', 'request_id': 'req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.095862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 876/1745] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:23 2022] PUT /identity/v3/projects/f9390af8e9f645c3b446a1bae563b0ae/users/97fedba50781434e892c22142aba4dab/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:24.097079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:24.122174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73dd0d77849e47b5847e97b655f442aa', 'request_id': 'req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.148733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73dd0d77849e47b5847e97b655f442aa', 'request_id': 'req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.183282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] The token cache is being invalidate because project 73dd0d77849e47b5847e97b655f442aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:24.183689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.184430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b5d9b050-7c3a-43d5-944b-f3e2644fa4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.186180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 870/1746] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:23 2022] DELETE /identity/v3/projects/73dd0d77849e47b5847e97b655f442aa => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:24.189013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.197001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.197395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.298833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] MFA Rules not processed for user `97fedba50781434e892c22142aba4dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:24.300820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:24.302774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591ab15-9a52-48ee-9304-8fdde6909475 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:24.332634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:24.335537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.335880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.336221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.336975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:24.348618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:24.407137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 877/1747] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:24.408590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:24.416118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:24.491634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb63c36-96e4-457c-bc87-fde16d314d7a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:24.506157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:24.508093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.508647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.509853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] PATH_INFO: `/v3/users/27480e60f0914a828abeea285741783d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.512237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27480e60f0914a828abeea285741783d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:24.515277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:24.599516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27480e60f0914a828abeea285741783d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.618599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27480e60f0914a828abeea285741783d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.619914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27480e60f0914a828abeea285741783d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.636470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27480e60f0914a828abeea285741783d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.637453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27480e60f0914a828abeea285741783d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:24.640289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:24.674713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 878/1748] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:24 2022] DELETE /identity/v3/users/27480e60f0914a828abeea285741783d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:24.675854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:24.678980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:24.837214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aceb39-4994-4e00-a490-eba899747765 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:24.851480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:24.852100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.852382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.852660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.853404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:24.855396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf733a5-3783-4555-a6c8-12ad677d7ef5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:24.894697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 879/1749] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:24.898987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc187-31f7-407e-ae1b-36ec7cc22e41 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.898987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc187-31f7-407e-ae1b-36ec7cc22e41 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.898987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc187-31f7-407e-ae1b-36ec7cc22e41 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:24.901249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 880/1750] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:19:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:24.902209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cc187-31f7-407e-ae1b-36ec7cc22e41 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:24.935180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cc187-31f7-407e-ae1b-36ec7cc22e41 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:24.955107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:24.956060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:24.956353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:24.956982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] PATH_INFO: `/v3/projects/16fbf99a7a254e1ea781b57898102f18/users/19b46569ce1c4f05af7954849ad7b60f/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.009382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16fbf99a7a254e1ea781b57898102f18, user_id=19b46569ce1c4f05af7954849ad7b60f, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.011070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 871/1751] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 824 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:25.016058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e8077-6acb-42f9-821e-203217219496 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.017094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.019475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:25.038567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80c807-aa0e-4c75-9cab-4a127f6fcf18 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.047761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.048795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.049169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.050187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.051623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.055022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.057292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 881/1752] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:24 2022] PUT /identity/v3/projects/16fbf99a7a254e1ea781b57898102f18/users/19b46569ce1c4f05af7954849ad7b60f/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:25.068628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.070002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.070757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.072606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:25.131697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 872/1753] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:25.136007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.137249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] MFA Rules not processed for user `19b46569ce1c4f05af7954849ad7b60f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:25.138598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:25.279815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 882/1754] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:25.283183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.287754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22d97b-a3b7-4305-aee2-395cb37896e6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.311504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.315804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.316346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45055649-b8a2-4037-be79-8aa72ab5a566 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.317039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.317536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.319732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.325253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.325713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.326732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.327429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.327855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.329272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.333289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b953ee7-a734-4a0c-a0e0-f828301b0a30 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.379481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 883/1755] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:25.382402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.383143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.383785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.394187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 873/1756] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/users => generated 303 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:25.394985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.416687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:25.418310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:25.421419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10329fac-3f1f-4191-8177-abf8ce9da456 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.441355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:25.531078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 884/1757] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:25.532213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.591672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.592983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.593489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.594234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.595971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.600618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.651939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c847015-396e-43c1-94c4-1e6f649d2fce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.664310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.665013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.665296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.665576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.666831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.668748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.682894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 885/1758] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:25.687966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.689961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:25.708468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 874/1759] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:25 2022] POST /identity/v3/users => generated 296 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:25.726397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.729280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954a301-efe5-4129-93e7-f82c6dfbfad4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.737592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:25.747993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.749012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.749408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.750970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] PATH_INFO: `/v3/users/e5a0348d621a4edf957492c9eea5d731` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.755166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a0348d621a4edf957492c9eea5d731)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.764151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.782068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792c05-d36c-4bbe-944c-b7a89bcc9bc9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.798066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.799552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.799836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.800111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.829735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.831373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:25.865154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 875/1760] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:25.870745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:25.892589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f525ecd-c929-4c6b-b120-8d85751968b6 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:25.907418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:25.910462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:25.911626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:25.911924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] PATH_INFO: `/v3/projects/29c1ab3ceab44218ab75fa8b348ec932/users/eb460257bb894249ab9058cfdfcc809d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:25.921708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a0348d621a4edf957492c9eea5d731'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:25.933519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a0348d621a4edf957492c9eea5d731'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:25.934803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a0348d621a4edf957492c9eea5d731'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:25.947085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a0348d621a4edf957492c9eea5d731'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:25.950942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a0348d621a4edf957492c9eea5d731'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:25.957496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c1ab3ceab44218ab75fa8b348ec932, user_id=eb460257bb894249ab9058cfdfcc809d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:25.968653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.005907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 886/1761] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:25 2022] DELETE /identity/v3/users/e5a0348d621a4edf957492c9eea5d731 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:26.007743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.013843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 876/1762] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:25 2022] PUT /identity/v3/projects/29c1ab3ceab44218ab75fa8b348ec932/users/eb460257bb894249ab9058cfdfcc809d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:26.014985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.020613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.021366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.100814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:26.111094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:26.186102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c51ed2-096f-4472-9f15-1eeed34b5c88 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.189029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb638a1-ebef-4961-876d-fb9c4276bab2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.203403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.204851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.205940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.206477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.207112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.207437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.208287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.208655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.210354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] PATH_INFO: `/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.213561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f2eac1de3e46d3b9966e1489fa4d82)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.215540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7d291-87bf-49be-a416-0058f7973c02 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.217597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.239518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 887/1763] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:26.242659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.244723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.245131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.293287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:26.295201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.305578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 888/1764] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:26.307672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.308339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.339872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f2eac1de3e46d3b9966e1489fa4d82', 'request_id': 'req-36832f2a-6097-439b-afb3-eec7a74f2cea'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:26.341941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ca13f-c973-4e52-aaa2-ff475fda2264 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.356068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f2eac1de3e46d3b9966e1489fa4d82', 'request_id': 'req-36832f2a-6097-439b-afb3-eec7a74f2cea'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:26.374545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f2eac1de3e46d3b9966e1489fa4d82', 'request_id': 'req-36832f2a-6097-439b-afb3-eec7a74f2cea'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:26.397381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] The token cache is being invalidate because project b5f2eac1de3e46d3b9966e1489fa4d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:26.397854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:26.398550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:26.400194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 877/1765] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:26 2022] DELETE /identity/v3/projects/b5f2eac1de3e46d3b9966e1489fa4d82 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:26.403900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.406550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.435142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.435984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.436340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.436690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.437770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.439687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.456345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 889/1766] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:26.457842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.460436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.523037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36832f2a-6097-439b-afb3-eec7a74f2cea admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.528791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.530328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.530802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.531214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] PATH_INFO: `/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d/users/3f1dd5876715440e99be8906aafb8eab/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.591013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebaffb-4dc7-4797-8a1e-4add6adf6a81 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.597651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.598391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.598682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.598961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.599400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9ca6dcc2c644baa1ac79628bb7fd1d, user_id=3f1dd5876715440e99be8906aafb8eab, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.604295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.606089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e6fd8-9650-4f10-b0c1-b11628e52d5d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.644272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.646290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.660451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 878/1767] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:26 2022] PUT /identity/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d/users/3f1dd5876715440e99be8906aafb8eab/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:26.660451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.660451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.660451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.666911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 890/1768] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:26.668240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.674980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.686112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:26.687467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:26.700912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de9636-41ea-4f7e-96e0-0b8befbb8d8b service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.771620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:26.807526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:26.839920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.842371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.843322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.843664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] PATH_INFO: `/v3/projects/29c1ab3ceab44218ab75fa8b348ec932/users/eb460257bb894249ab9058cfdfcc809d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.850806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 879/1769] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:26.851931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.852035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c1ab3ceab44218ab75fa8b348ec932, user_id=eb460257bb894249ab9058cfdfcc809d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.858461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b3c14-e4f9-4545-9ff5-182d6ee7196b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.865757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec22737-605a-4906-b96c-5ed0eaec7853 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.875057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:26.876192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.876565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.877561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.878556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:26.882012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:26.903314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 891/1770] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:26 2022] PUT /identity/v3/projects/29c1ab3ceab44218ab75fa8b348ec932/users/eb460257bb894249ab9058cfdfcc809d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:26.909318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66e61-5eda-46a9-b815-914650016e5a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:26.909318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66e61-5eda-46a9-b815-914650016e5a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:26.909318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f66e61-5eda-46a9-b815-914650016e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:26.910164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 880/1771] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:26.911636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:26.925049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901404e-3e1a-439e-87b5-85329464c56c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:26.969915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-59f66e61-5eda-46a9-b815-914650016e5a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:26.971169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-59f66e61-5eda-46a9-b815-914650016e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.065218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:27.066793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.067271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.069628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] PATH_INFO: `/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d/users/3f1dd5876715440e99be8906aafb8eab/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.073389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9ca6dcc2c644baa1ac79628bb7fd1d, user_id=3f1dd5876715440e99be8906aafb8eab, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:27.080754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5029-4ec9-464f-b337-0adac4206db3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:27.114383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 881/1772] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:26 2022] PUT /identity/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d/users/3f1dd5876715440e99be8906aafb8eab/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:27.116555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.116843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.117124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.128912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 892/1773] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:27.130609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5d40-57e9-4cdc-8327-e4d5dbf71039 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.131860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5d40-57e9-4cdc-8327-e4d5dbf71039 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.133452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5d40-57e9-4cdc-8327-e4d5dbf71039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.177965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] MFA Rules not processed for user `eb460257bb894249ab9058cfdfcc809d`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:27.181366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.213849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a72a5d40-57e9-4cdc-8327-e4d5dbf71039 None None] MFA Rules not processed for user `3f1dd5876715440e99be8906aafb8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:27.216134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a72a5d40-57e9-4cdc-8327-e4d5dbf71039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.302323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 882/1774] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:27.304089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:27.325863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 893/1775] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:27.327290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.327595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.328336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.352104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] MFA Rules not processed for user `3cf837e2ab5b4e58844a1023cf5f9237`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:27.353339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.366481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:27.383119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85b9e-3685-4556-a4bf-26f3866f8b52 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:27.388249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:27.388924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.389207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.389484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.390250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:27.390753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:27.392558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:27.405221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 883/1776] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:27.406081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:27.407771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.422283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 894/1777] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:27.423418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:27.424671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.438539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3efb0-f30e-459f-8d3b-f88243482403 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:27.446469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:27.446469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.446469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.446469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] PATH_INFO: `/v3/projects/892ea92d619d406d9f4e350f008c3fac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.448637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=892ea92d619d406d9f4e350f008c3fac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:27.449783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:27.471827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:27.544689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892ea92d619d406d9f4e350f008c3fac', 'request_id': 'req-8af2e2c3-adcd-4701-b723-7ed395dfd722'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:27.551877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892ea92d619d406d9f4e350f008c3fac', 'request_id': 'req-8af2e2c3-adcd-4701-b723-7ed395dfd722'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:27.571753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b8268-34d3-4089-b773-8c39f22acb3c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:27.580224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892ea92d619d406d9f4e350f008c3fac', 'request_id': 'req-8af2e2c3-adcd-4701-b723-7ed395dfd722'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:27.613456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] The token cache is being invalidate because project 892ea92d619d406d9f4e350f008c3fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:27.613767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:27.614548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:27.617951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 895/1778] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:27 2022] DELETE /identity/v3/projects/892ea92d619d406d9f4e350f008c3fac => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:27.619014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:27.624299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.650748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:27.651950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.652510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.652888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.655191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.767884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:27.769928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:27.772636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2e2c3-adcd-4701-b723-7ed395dfd722 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:27.777961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:27.778930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:27.779595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:27.780036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:27.782888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.791569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 884/1779] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:27.794080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:27.797931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:27.952152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:27.965472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:27.968936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b237-8e49-4a3b-be65-450aa47ef341 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:27.986874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac829-158f-42b8-a61e-690bda6eb4bc service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:27.998352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 896/1780] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:27.999988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222424-c4af-4349-a682-acf85946b1fe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.000286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222424-c4af-4349-a682-acf85946b1fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.000622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222424-c4af-4349-a682-acf85946b1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.001571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.002528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.002973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.003489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.004551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.007030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.062029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 885/1781] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:28.063667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.069458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.081620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-da222424-c4af-4349-a682-acf85946b1fe None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:28.082967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-da222424-c4af-4349-a682-acf85946b1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.181356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:28.215288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 897/1782] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:28.215288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222424-c4af-4349-a682-acf85946b1fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.231667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b49ff1-bebb-487e-ad07-97f43d2f82dc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.237891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.238676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.238963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.239282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.248995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.252852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222424-c4af-4349-a682-acf85946b1fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.284726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.294996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.294996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.294996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.294996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.303681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815f8a6-f538-4e06-ac1a-781ad5daebff admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.347529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 898/1783] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:28 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:28.352204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aeddf-1178-478c-b175-55ab4ae319c5 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.355131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aeddf-1178-478c-b175-55ab4ae319c5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.355968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aeddf-1178-478c-b175-55ab4ae319c5 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.371292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 899/1784] 10.222.0.39 () {54 vars in 912 bytes} [Thu May 26 03:19:28 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:28.378669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.378981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.379436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.400811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.407067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.429620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:28.433410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.439215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 886/1785] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:28.441285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.455957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 900/1786] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:28.462296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced549-708c-4183-87bb-d5623c934dd7 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.463588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.475142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.476573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.477049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.477650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.479421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3b6bf-a7c9-4d3f-8c41-2609741f7185 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.484989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.487026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.488159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.490112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.490112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.490986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.494178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038e320-e59f-4549-a9aa-6a0b278b4a41 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.521605 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 901/1787] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:28.524498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.527712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.528414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.585827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:28.587183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.660593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.668170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.742155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 887/1788] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:28.754001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.786804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 902/1789] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:19:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:28.789162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.842821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c02be-6ec3-4cf8-b684-6ac3e01771e4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.857784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76231acb-47bd-4616-8d05-a4ab1e4be539 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.865721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.867269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.868847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.869614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.873051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548414-e630-4176-938f-391a812795f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.873505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.876407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548414-e630-4176-938f-391a812795f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.877659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548414-e630-4176-938f-391a812795f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.878261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50548414-e630-4176-938f-391a812795f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.879333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.880641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548414-e630-4176-938f-391a812795f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:28.882361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548414-e630-4176-938f-391a812795f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.886067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50548414-e630-4176-938f-391a812795f7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.909569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 903/1790] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:28.910754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.916795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 888/1791] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:28.918153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548414-e630-4176-938f-391a812795f7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:28.922445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:28.952583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ac19-e16f-43b9-9b15-4af6499a3dbd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.954018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50548414-e630-4176-938f-391a812795f7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:28.963165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.963746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.964146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.964477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.980002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:28.980452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:28.982142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:28.982800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:28.984187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:28.984922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] PATH_INFO: `/v3/projects/b8b974e9951b40798336cf9444b65ce9/users/33ce42bf0ba24c30a0932e9f4e1cc530/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:28.996568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 904/1792] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:28.997424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.015963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cba683-75b0-4d7a-bc9c-aa05b0f938d3 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.025571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.027781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.028111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.028299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:29.029401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.032286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.034454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8b974e9951b40798336cf9444b65ce9, user_id=33ce42bf0ba24c30a0932e9f4e1cc530, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.036955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.038597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02bd6a-74be-4c33-9adc-cb5f180db1c2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.128383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 889/1793] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:28 2022] PUT /identity/v3/projects/b8b974e9951b40798336cf9444b65ce9/users/33ce42bf0ba24c30a0932e9f4e1cc530/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:29.145475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 905/1794] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:28 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:29.147630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.152335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.157437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.157755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.250191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] MFA Rules not processed for user `33ce42bf0ba24c30a0932e9f4e1cc530`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:29.251469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:29.371868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 890/1795] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:29.375697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.402682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40123ad-e224-4055-9d25-4dbadbe0447b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.416241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.417084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.418667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.419378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.421437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.437398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.487784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 906/1796] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/users => generated 286 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:29.489106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.525085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f38619-ee0c-4dbd-bf53-b0f02b5ca7bb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.547910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.547910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.547910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.548276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.548930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.550773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.567836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:29.582292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ffc1-b920-4be3-89ff-89c23e349ee1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.594786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 907/1797] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:29.599123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.607807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.608542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.608822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.609096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.609806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.611775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.622309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9af4-600e-46ea-8bc2-008cd8cb05d7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.629936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.633108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.633108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.633108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.634778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.637720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.663906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 908/1798] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:29.666409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.681010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3e8e-ca13-4bfa-ace1-b24ec524c9fb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.688208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:29.689304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.689928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.689928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] PATH_INFO: `/v3/projects/74032b07d42e429c85df69f2c4a3d72b/users/1039efdb9a874b008b2d25b85d3ba510/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.698593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 891/1799] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/projects => generated 379 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:29.712699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:29.714301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74032b07d42e429c85df69f2c4a3d72b, user_id=1039efdb9a874b008b2d25b85d3ba510, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:29.718157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022deb-a47e-44e7-92ca-ef8af67d11a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:29.754529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 909/1800] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:29 2022] PUT /identity/v3/projects/74032b07d42e429c85df69f2c4a3d72b/users/1039efdb9a874b008b2d25b85d3ba510/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:29.756313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:29.757696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:29.758018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:29.806950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] MFA Rules not processed for user `1039efdb9a874b008b2d25b85d3ba510`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:29.808899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:29.929352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232eb68-d656-4ca2-af5e-855648836a57 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:29.956586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:29.975788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 910/1801] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:29.980097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.052694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.054326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.054717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.055180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.056155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.058305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.069646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e1a60-0116-4f36-b899-3ecc49f1cc2d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.078728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.079445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.079777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.080051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.080771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.082898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.089409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 892/1802] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/users => generated 290 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:30.097602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.111487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013ac7-9ca5-4a26-8926-44e400dcfeaf admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.117967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.118739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.119017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.119336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.120061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.120455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 911/1803] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:29 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:30.121683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.123992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.135932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 893/1804] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:30.139591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.156221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1c89-a443-48a9-8a23-02d713e9550c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.179484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:30.231658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6df9d-11a2-4af2-855b-6c36a8ec457b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.242341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.243341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.243704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.244055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.244827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.246938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.250712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.253787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.254372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.254806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.255957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.258414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.300597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 912/1805] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:30 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:30.304958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.321673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 894/1806] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:30.322744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29c29-637f-408a-adc2-3e2877fdf49b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.325753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.332951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.334017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.335177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.335822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.336785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.338801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.345766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d60fd3-2b5e-4a09-958b-e1691e4750a8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.356141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 913/1807] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:30.357955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.359915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.360945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.361391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.361811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] PATH_INFO: `/v3/projects/7b8914a09ffc4281aabdbcac6b0baaaa/users/f3f5e7d0de4c474c9d767777aaef477b/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.391767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15f623-b3dd-49df-8648-4849dac3c730 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.394529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8914a09ffc4281aabdbcac6b0baaaa, user_id=f3f5e7d0de4c474c9d767777aaef477b, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.397363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.398131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9646f-75eb-48ac-84e4-b88a24f3b38e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.398245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.398522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.398801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] PATH_INFO: `/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165/users/125f89e5e3b3400cab3a01e809f2618a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.413476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 895/1808] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:30 2022] PUT /identity/v3/projects/7b8914a09ffc4281aabdbcac6b0baaaa/users/f3f5e7d0de4c474c9d767777aaef477b/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:30.423701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.423993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.424267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.425505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb9c2d0ac864ee9a51841ab5498d165, user_id=125f89e5e3b3400cab3a01e809f2618a, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.429455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.452125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 914/1809] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:30 2022] PUT /identity/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165/users/125f89e5e3b3400cab3a01e809f2618a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:30.455621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.457263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] MFA Rules not processed for user `f3f5e7d0de4c474c9d767777aaef477b`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:30.458501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:30.542119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:30.555292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cb97-8fee-4179-89a5-0603677dc348 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.560684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.561360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.561779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.562058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.562815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.571291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 896/1810] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:30.574340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.575125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.597219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f079-4f55-4e1a-91dd-82b135f0fa1c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.597852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 915/1811] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:30.602465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.611620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.612296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.614411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.614703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.618610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c778e-ae21-4cba-818f-a2f018cc7dce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.620438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.622740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.630944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.633044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.633549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.634083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] PATH_INFO: `/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165/users/125f89e5e3b3400cab3a01e809f2618a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.637721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb9c2d0ac864ee9a51841ab5498d165, user_id=125f89e5e3b3400cab3a01e809f2618a, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.643269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da10511-e086-412a-800e-6a04819f1f52 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.675646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 916/1812] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:30 2022] PUT /identity/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165/users/125f89e5e3b3400cab3a01e809f2618a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:30.680903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 897/1813] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:30 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:30.686444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.687503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.687878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.688087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.758924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] MFA Rules not processed for user `125f89e5e3b3400cab3a01e809f2618a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:30.761263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:30.780777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8a35-4530-4fe9-9101-d64f4787e116 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.786828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.788352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.788714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.789064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.790067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.792680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.824158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 898/1814] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:30 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:30.825211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.842273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81f43-c455-40ec-8a74-babff1571041 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.849595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.850870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.851151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.851461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.852200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.853960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.867278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 899/1815] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:30.869437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.882510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40e1f-bcbb-4321-876d-4694c885b0c1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:30.888662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:30.889754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:30.890366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:30.890793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] PATH_INFO: `/v3/projects/84be9b0cbe6648f9ac00d605bc511682/users/9769cdc434184008810075910559a2ae/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:30.900614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:30.909416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] RBAC: Authorizing `identity:create_grant(project_id=84be9b0cbe6648f9ac00d605bc511682, user_id=9769cdc434184008810075910559a2ae, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:30.912150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:30.917888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 917/1816] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:30.931905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 900/1817] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:30 2022] PUT /identity/v3/projects/84be9b0cbe6648f9ac00d605bc511682/users/9769cdc434184008810075910559a2ae/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:30.933059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:30.944440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ea512-69c9-48a2-b0a6-b806ce7edee9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.012500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.013311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.013629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.013908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.014642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.016578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794a71b-2256-4b30-82e5-931f97b2b66d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.034714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 918/1818] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:31.035935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:31.047692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db48a31-de54-41ab-a261-12060b1bdaba admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.054304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.054827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.055134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.055499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] PATH_INFO: `/v3/projects/84be9b0cbe6648f9ac00d605bc511682/users/9769cdc434184008810075910559a2ae/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.057794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84be9b0cbe6648f9ac00d605bc511682, user_id=9769cdc434184008810075910559a2ae, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.060733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.086089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 901/1819] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:31 2022] PUT /identity/v3/projects/84be9b0cbe6648f9ac00d605bc511682/users/9769cdc434184008810075910559a2ae/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:31.089871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.090162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.091015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.147092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] MFA Rules not processed for user `9769cdc434184008810075910559a2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:31.147397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:31.149662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:31.241606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9fe1-40d3-4c11-a1a3-4be01a76252a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.247989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.249137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.249494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.249912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.282795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 919/1820] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:31.286469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:31.330972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.332476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.344142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 902/1821] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:31.398326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:31.418122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621150e5-11e6-43c7-bcfb-e05c0c1b2b39 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.421844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.422614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.422961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.424485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.434201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.436066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.446953 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 920/1822] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:31.534937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:31.546624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0f13a-8286-4f2b-b57f-432e8cc2c00c service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.554903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.555679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.556024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.556364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.589183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:31.599026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bc103-0d79-4f4a-8ecf-fd7d887bbc0b service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:31.603392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:31.604040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:31.604307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:31.604580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:31.617167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.618833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.627916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 903/1823] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:31.664718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:31.666182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af681751-e52c-4c07-b5c5-87e073055cac service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:31.675113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 921/1824] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:19:35.356082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:35.357666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:35.471484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f00fab-4290-4fc8-96f5-2ea4db202095 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:35.471484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:35.471484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:35.471484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:35.471806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] PATH_INFO: `/v3/users/97fedba50781434e892c22142aba4dab` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:35.472722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fedba50781434e892c22142aba4dab)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:35.475493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:35.586509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fedba50781434e892c22142aba4dab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:35.587471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fedba50781434e892c22142aba4dab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:35.605321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fedba50781434e892c22142aba4dab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:35.609187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fedba50781434e892c22142aba4dab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:35.611772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:35.625777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fedba50781434e892c22142aba4dab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:35.656888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 904/1825] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:35 2022] DELETE /identity/v3/users/97fedba50781434e892c22142aba4dab => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:35.661105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:35.661578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:35.661650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:35.669091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:35.686539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:35.688081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:35.769959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 922/1826] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:35.780567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:35.782631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:35.881666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6befa-c0e9-4b9f-8f3c-45459d677b0c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:35.887335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:35.888348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:35.888871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:35.889260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:35.914200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:35.915836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:35.944947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 905/1827] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:36.256911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:36.258467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:36.273716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9c706-11ee-4bc6-9b9e-40de3656006d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:36.278517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:36.279584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:36.280166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:36.280644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] PATH_INFO: `/v3/projects/f9390af8e9f645c3b446a1bae563b0ae` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:36.282745 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9390af8e9f645c3b446a1bae563b0ae)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:36.287171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:36.337650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9390af8e9f645c3b446a1bae563b0ae', 'request_id': 'req-058826cf-5581-4b3c-b54f-14a98c6befe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.342942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9390af8e9f645c3b446a1bae563b0ae', 'request_id': 'req-058826cf-5581-4b3c-b54f-14a98c6befe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.349120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9390af8e9f645c3b446a1bae563b0ae', 'request_id': 'req-058826cf-5581-4b3c-b54f-14a98c6befe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.362786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] The token cache is being invalidate because project f9390af8e9f645c3b446a1bae563b0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:36.363074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.363735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.366175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 923/1828] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:36 2022] DELETE /identity/v3/projects/f9390af8e9f645c3b446a1bae563b0ae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:36.369817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:36.380051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf509786-ffaa-4fc7-9baa-313b42d0e842 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:36.440833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:36.441712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:36.441993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:36.442270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] PATH_INFO: `/v3/users/eb460257bb894249ab9058cfdfcc809d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:36.444199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb460257bb894249ab9058cfdfcc809d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:36.446144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:36.490668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb460257bb894249ab9058cfdfcc809d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.491350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb460257bb894249ab9058cfdfcc809d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.498144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb460257bb894249ab9058cfdfcc809d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.498771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb460257bb894249ab9058cfdfcc809d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.511139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63a58576-ee74-4e58-9b34-4a7d2949b052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb460257bb894249ab9058cfdfcc809d'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.519973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:36.525114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 906/1829] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:36 2022] DELETE /identity/v3/users/eb460257bb894249ab9058cfdfcc809d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:36.799675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:36.801121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:36.814639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058826cf-5581-4b3c-b54f-14a98c6befe6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:36.873549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:36.874318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:36.874867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:36.875144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] PATH_INFO: `/v3/projects/29c1ab3ceab44218ab75fa8b348ec932` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:36.876338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c1ab3ceab44218ab75fa8b348ec932)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:36.878262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:36.919116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:36.937249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c1ab3ceab44218ab75fa8b348ec932', 'request_id': 'req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.942741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c1ab3ceab44218ab75fa8b348ec932', 'request_id': 'req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.949217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c1ab3ceab44218ab75fa8b348ec932', 'request_id': 'req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.962197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] The token cache is being invalidate because project 29c1ab3ceab44218ab75fa8b348ec932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:36.962480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.963096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-396e9e25-cdb0-4b6f-af64-3cb17b21b1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:36.964658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 924/1830] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:36 2022] DELETE /identity/v3/projects/29c1ab3ceab44218ab75fa8b348ec932 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:36.970133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:36.970418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:36.970715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:36.989125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:36.990847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:37.056474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 907/1831] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:37.063878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.064167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.064444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.099657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:37.101056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:37.109787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 925/1832] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:37.113681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.124415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42ddb0-a78d-4578-a74f-f65ef55dc47c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.129198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.129866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.130150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.130428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.131233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:37.131611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.133294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.146231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 908/1833] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:37.151060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.162105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b91d-812c-42b9-9b7b-616b7b21da18 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.167186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.168472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.168968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.169345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.170154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.172229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.215150 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 926/1834] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/projects => generated 399 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:37.216456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.226377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed103a-f2dd-4307-a66a-fb564a3a798c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.287935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.288634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.288915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.289190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.289879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.291871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.316459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 909/1835] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:37.317777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.328731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b26dc-b7fd-4226-bee7-f1132b94dc8b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.333534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.334341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.334624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.334892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.335648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.337289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.348016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 927/1836] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:37.352489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.362500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f99bb-b1f8-4a7f-bd3f-e408039f9865 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.368094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.368764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.369109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.369316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.370049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.372319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e877bf-0526-403c-87e8-b3127397129e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.386219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 910/1837] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:37.387540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.399090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbe859-c28b-44ce-84b9-277766188e74 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.403892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.404823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.405107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.405382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] PATH_INFO: `/v3/projects/f8533a97f5c14200ada0a684f7282afa/users/3ab60193d6be4dfba48c484a1378176f/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.420267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8533a97f5c14200ada0a684f7282afa, user_id=3ab60193d6be4dfba48c484a1378176f, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.422925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.435328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 928/1838] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:37 2022] PUT /identity/v3/projects/f8533a97f5c14200ada0a684f7282afa/users/3ab60193d6be4dfba48c484a1378176f/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:37.445992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.446305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.446583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.472950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] MFA Rules not processed for user `3ab60193d6be4dfba48c484a1378176f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:37.474150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:37.530803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:37.540166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 911/1839] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:37.544315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.605842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2c6c5-5eae-4a99-b8cd-2bacaedad63e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.610542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.611193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.611513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.611794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.612487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.614390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.640102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 929/1840] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/projects => generated 399 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:37.644376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.655541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163e980-08f3-4a8e-8c77-31e77e6ce59b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.713214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.713892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.714200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.714476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.715496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.717415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.737056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 912/1841] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:37 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:37.742525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.755307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac96a6-4dae-488a-ad72-9a56f864bb78 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.760890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.761581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.762638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.762942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.763710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.765708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.777927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 930/1842] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:37.782655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.793409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b256419-2976-4c58-b474-4188e711be54 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.798387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.799299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.799585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.799857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] PATH_INFO: `/v3/projects/a75e8735391a43888b95675b3a3e7dd7/users/359eba528a824635867677f19be44d75/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.815315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75e8735391a43888b95675b3a3e7dd7, user_id=359eba528a824635867677f19be44d75, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.818044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.833292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 913/1843] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:37 2022] PUT /identity/v3/projects/a75e8735391a43888b95675b3a3e7dd7/users/359eba528a824635867677f19be44d75/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:37.838470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.850375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07676421-357e-4329-8ffa-d4a328273180 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.889082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:37.931315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.932005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.932291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.932617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.933350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.935295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497991a0-d37b-43f6-ac87-ea9c5f256dbd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:37.948378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 931/1844] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:37.953140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:37.967180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb208aa-0f35-4940-a799-bc0a2bcee288 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:37.978616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:37.981487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:37.982694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:37.982694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] PATH_INFO: `/v3/projects/a75e8735391a43888b95675b3a3e7dd7/users/359eba528a824635867677f19be44d75/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:37.984502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75e8735391a43888b95675b3a3e7dd7, user_id=359eba528a824635867677f19be44d75, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:37.988206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:38.007288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 914/1845] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:37 2022] PUT /identity/v3/projects/a75e8735391a43888b95675b3a3e7dd7/users/359eba528a824635867677f19be44d75/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:38.017628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:38.018514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:38.019650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:38.054351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] MFA Rules not processed for user `359eba528a824635867677f19be44d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:38.055857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:38.168473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 932/1846] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:38.175319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:38.177074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:38.275398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10656d7-d511-4085-b57d-84073fabe5fb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:38.281692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:38.282139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:38.282430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:38.284237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:38.318070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:38.321603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32229b90-7614-4119-82fd-43482be4733e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:38.345097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 915/1847] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:38.470836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:38.481655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806356a3-9104-491f-a34e-a5a4019d7ce5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:38.485768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:38.486432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:38.486714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:38.486992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:38.585851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:38.587350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550c571-c14d-4314-9ebf-1fc75f935e39 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:38.623359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 933/1848] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:41.359257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:41.359904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:41.359904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:41.364700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 916/1849] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:19:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:19:41.379992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:41.381346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:41.381879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:41.418652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:41.420143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:41.495419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:41.528380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 934/1850] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:19:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:41.544676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:41.547080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:41.579122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6636ff-0733-4263-8d30-68ceabe820c7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:41.588790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:41.594100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:41.594100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:41.594100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:41.611947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:41.614377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:41.616767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:41.620901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea8e46-9f14-49e8-bbea-f78e90f9fe8f service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:41.640869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 917/1851] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:41.713093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbb163-fa22-47ad-8428-fc9eb0fbac86 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:41.718279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:41.719169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:41.719484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:41.719759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] PATH_INFO: `/v3/users/33ce42bf0ba24c30a0932e9f4e1cc530` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:41.721262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ce42bf0ba24c30a0932e9f4e1cc530)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:41.723129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:41.820346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ce42bf0ba24c30a0932e9f4e1cc530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:41.828213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ce42bf0ba24c30a0932e9f4e1cc530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:41.829060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ce42bf0ba24c30a0932e9f4e1cc530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:41.838810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ce42bf0ba24c30a0932e9f4e1cc530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:41.840587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ce42bf0ba24c30a0932e9f4e1cc530'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:41.859828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 935/1852] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:41 2022] DELETE /identity/v3/users/33ce42bf0ba24c30a0932e9f4e1cc530 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:41.864211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:41.864496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:41.864778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:41.886638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:41.888356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:41.951541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:41.991472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 918/1853] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:42.001419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:42.002941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:42.129140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1244f-e825-4187-8dc3-c193766e8f15 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:42.134126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:42.135090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:42.136053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:42.136732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:42.149749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:42.152051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:42.170667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 936/1854] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:42.490097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:42.491686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:42.503743 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c50249-8161-46e8-9a53-caa2fc830b3d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:42.509213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:42.510185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:42.510540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:42.510823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] PATH_INFO: `/v3/projects/b8b974e9951b40798336cf9444b65ce9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:42.512125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8b974e9951b40798336cf9444b65ce9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:42.514203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:42.562541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8b974e9951b40798336cf9444b65ce9', 'request_id': 'req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.569414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8b974e9951b40798336cf9444b65ce9', 'request_id': 'req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.576846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8b974e9951b40798336cf9444b65ce9', 'request_id': 'req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.587844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] The token cache is being invalidate because project b8b974e9951b40798336cf9444b65ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:42.588130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.588832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.590312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 919/1855] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:42 2022] DELETE /identity/v3/projects/b8b974e9951b40798336cf9444b65ce9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:42.594249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:42.599405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:42.605262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1b55-b9a2-4f42-837a-95c11049182e service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:42.669179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:42.670060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:42.670338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:42.671883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] PATH_INFO: `/v3/users/f3f5e7d0de4c474c9d767777aaef477b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:42.671883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f5e7d0de4c474c9d767777aaef477b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:42.673723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:42.725666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f5e7d0de4c474c9d767777aaef477b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.732410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f5e7d0de4c474c9d767777aaef477b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.733057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f5e7d0de4c474c9d767777aaef477b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.739813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f5e7d0de4c474c9d767777aaef477b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.740504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f5e7d0de4c474c9d767777aaef477b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:42.758195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 937/1856] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:42 2022] DELETE /identity/v3/users/f3f5e7d0de4c474c9d767777aaef477b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:43.041693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:43.043410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:43.064430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4554-64ed-4c57-9ce4-9c70fecbfa4f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:43.143519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:43.144604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:43.144794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:43.145072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] PATH_INFO: `/v3/projects/7b8914a09ffc4281aabdbcac6b0baaaa` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:43.147210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b8914a09ffc4281aabdbcac6b0baaaa)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:43.150197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:43.253454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b8914a09ffc4281aabdbcac6b0baaaa', 'request_id': 'req-c127fc22-6da3-473f-947e-3e02e321a87b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.257073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b8914a09ffc4281aabdbcac6b0baaaa', 'request_id': 'req-c127fc22-6da3-473f-947e-3e02e321a87b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.266683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b8914a09ffc4281aabdbcac6b0baaaa', 'request_id': 'req-c127fc22-6da3-473f-947e-3e02e321a87b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.268769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:43.279667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] The token cache is being invalidate because project 7b8914a09ffc4281aabdbcac6b0baaaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:43.279953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.280584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.282120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 920/1857] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:43 2022] DELETE /identity/v3/projects/7b8914a09ffc4281aabdbcac6b0baaaa => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:43.287452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:43.288939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:43.304140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce06c-037c-4605-bf41-4019caec1647 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:43.371284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:43.372178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:43.372528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:43.372876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] PATH_INFO: `/v3/users/9769cdc434184008810075910559a2ae` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:43.374136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9769cdc434184008810075910559a2ae)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:43.376211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:43.401916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:43.426307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9769cdc434184008810075910559a2ae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.433794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9769cdc434184008810075910559a2ae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.434476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9769cdc434184008810075910559a2ae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.441976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9769cdc434184008810075910559a2ae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.442590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdf592b4-f79f-488e-814b-016b24ec8f73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9769cdc434184008810075910559a2ae'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.469736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 938/1858] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:43 2022] DELETE /identity/v3/users/9769cdc434184008810075910559a2ae => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:43.740590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:43.742055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:43.755024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127fc22-6da3-473f-947e-3e02e321a87b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:43.812399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:43.813256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:43.813534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:43.813806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] PATH_INFO: `/v3/projects/84be9b0cbe6648f9ac00d605bc511682` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:43.814956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84be9b0cbe6648f9ac00d605bc511682)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:43.816890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:43.861597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84be9b0cbe6648f9ac00d605bc511682', 'request_id': 'req-88fb00e0-a70f-4262-984f-bd4f461e4d6f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.868126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84be9b0cbe6648f9ac00d605bc511682', 'request_id': 'req-88fb00e0-a70f-4262-984f-bd4f461e4d6f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.875084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84be9b0cbe6648f9ac00d605bc511682', 'request_id': 'req-88fb00e0-a70f-4262-984f-bd4f461e4d6f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.885963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] The token cache is being invalidate because project 84be9b0cbe6648f9ac00d605bc511682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:43.886246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.886856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-88fb00e0-a70f-4262-984f-bd4f461e4d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:43.888405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 921/1859] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:43 2022] DELETE /identity/v3/projects/84be9b0cbe6648f9ac00d605bc511682 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:43.894486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:43.894993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:43.895082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:43.913174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:43.914352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:43.972254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 939/1860] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:43.980772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:43.981061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:43.981338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:43.999753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:44.000948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:44.009486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 922/1861] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:44.013702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.023620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde0e7c-c483-4ea5-8018-aca942a1cd4a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.028083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.028724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.029004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.029276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.029992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:44.030293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.031918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.043575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 940/1862] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:44.047937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.057511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9edf27-7d5e-46cb-8e62-0c378c730316 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.062106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.062746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.063029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.063335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.064031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.065932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.090991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 923/1863] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:44 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:44.095360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.105847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c4d5b-73a1-461a-826c-42770d7e3815 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.173352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.175662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.175956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.176399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.177380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.179885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.203685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 941/1864] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:44 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:44.207938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.220053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83ea0-e085-41e9-8740-38ebda9e12d2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.225760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.226580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.226867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.227197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.228050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.230239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.243591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 924/1865] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:44.247948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.265312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9674bb-6398-495e-8326-523155d8dee2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.285186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.285996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.286383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.286929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.287811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.289531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdd80a-1f36-486b-b4f0-5a17b38c5371 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.309002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 942/1866] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:44.310416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.330677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee091aa-b3c9-43c2-8e0f-b826ccc070e6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.339846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.342109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.342461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.342973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] PATH_INFO: `/v3/projects/3ea146d4a6eb4fe5b485a1b95a6b7b5f/users/1fd1e37f5f9d449c8fd5277a8fa01cfc/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.367314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea146d4a6eb4fe5b485a1b95a6b7b5f, user_id=1fd1e37f5f9d449c8fd5277a8fa01cfc, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.368671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.394187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 925/1867] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:44 2022] PUT /identity/v3/projects/3ea146d4a6eb4fe5b485a1b95a6b7b5f/users/1fd1e37f5f9d449c8fd5277a8fa01cfc/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:44.409976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381b78-229c-4219-a738-75884b37af01 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.410892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381b78-229c-4219-a738-75884b37af01 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.411208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381b78-229c-4219-a738-75884b37af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.429424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:44.453523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-56381b78-229c-4219-a738-75884b37af01 None None] MFA Rules not processed for user `1fd1e37f5f9d449c8fd5277a8fa01cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:44.454833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-56381b78-229c-4219-a738-75884b37af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:44.548170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.548650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 943/1868] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:44.582961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:44.732885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a129-96e7-4753-93e1-e274e59f1010 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.753514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.757305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.757672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.757958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.758690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.773089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:44.839587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 926/1869] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:44 2022] POST /identity/v3/projects => generated 381 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:44.843542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56381b78-229c-4219-a738-75884b37af01 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:44.857590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56381b78-229c-4219-a738-75884b37af01 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:44.951147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:44.952567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:44.953157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:44.953659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:44.954493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:44.960297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.049046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 944/1870] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:44 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:45.050232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.067608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd9d2e-20e6-4baf-87e2-29296ade14a1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.076565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.077450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.081917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.082301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.083169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.085101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.110457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.111355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 927/1871] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:45.125568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7519a-0432-4ebd-b663-b5a1cd32385e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.132201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.133314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.137553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.137553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] PATH_INFO: `/v3/projects/b6e96b59eac744bfbc06f522201cf2d2/users/0554d1f0ffb249cf94a8c5a10d722451/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.154053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e96b59eac744bfbc06f522201cf2d2, user_id=0554d1f0ffb249cf94a8c5a10d722451, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.155922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.169643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 945/1872] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:45 2022] PUT /identity/v3/projects/b6e96b59eac744bfbc06f522201cf2d2/users/0554d1f0ffb249cf94a8c5a10d722451/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:45.174585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.185520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a05a7-b22c-45a1-93f2-0f2f05d571f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.258425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.259099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.259415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.260313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.261047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.262637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834527a-5a4e-4214-b901-7da4c5263521 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.273086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 928/1873] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:45.277268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.287872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d807b-4a63-4315-b422-b3023c69d5ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.292702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.293735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.294082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.294558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] PATH_INFO: `/v3/projects/b6e96b59eac744bfbc06f522201cf2d2/users/0554d1f0ffb249cf94a8c5a10d722451/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.296622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e96b59eac744bfbc06f522201cf2d2, user_id=0554d1f0ffb249cf94a8c5a10d722451, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.299331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.312744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 946/1874] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:45 2022] PUT /identity/v3/projects/b6e96b59eac744bfbc06f522201cf2d2/users/0554d1f0ffb249cf94a8c5a10d722451/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:45.326599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.327562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.327917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.361471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] MFA Rules not processed for user `0554d1f0ffb249cf94a8c5a10d722451`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:45.367139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:45.449693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 929/1875] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:45.467073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.468865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:45.525764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:45.530017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51879917-c26e-4229-9aeb-ed384409b101 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.535219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.536002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.536393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.536878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.546069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.547646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7bb5f-202e-43ce-9d3d-2930ff107fe6 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.559531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 947/1876] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:45.614426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.616638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:45.648362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:45.711006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e4ed5-fceb-4544-bd7f-b9cd9431ab0f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:45.718114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:45.719002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.719307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.719591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] PATH_INFO: `/v3/users/3ab60193d6be4dfba48c484a1378176f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.720811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab60193d6be4dfba48c484a1378176f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:45.722804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:45.823131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab60193d6be4dfba48c484a1378176f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:45.828424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab60193d6be4dfba48c484a1378176f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:45.839719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab60193d6be4dfba48c484a1378176f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:45.841225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab60193d6be4dfba48c484a1378176f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:45.859312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab60193d6be4dfba48c484a1378176f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:45.885514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 930/1877] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:45 2022] DELETE /identity/v3/users/3ab60193d6be4dfba48c484a1378176f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:45.887170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:45.887582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:45.887971 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:45.919841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:45.921204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:45.954929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:45.956530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.052022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b784e-6ea4-461e-8a28-8e44012813cc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.059177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.060141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.060422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.060693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] PATH_INFO: `/v3/users/19b46569ce1c4f05af7954849ad7b60f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.061964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b46569ce1c4f05af7954849ad7b60f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.064176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.070423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 948/1878] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:46.082065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.083711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.111713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:46.126476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b46569ce1c4f05af7954849ad7b60f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.128073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b46569ce1c4f05af7954849ad7b60f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.136641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b46569ce1c4f05af7954849ad7b60f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.138100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b46569ce1c4f05af7954849ad7b60f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.157364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b46569ce1c4f05af7954849ad7b60f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.159032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc03da6-45d0-45b7-986d-020644869f01 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.164047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.165350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.165635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.165907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.167868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.176493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 931/1879] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:45 2022] DELETE /identity/v3/users/19b46569ce1c4f05af7954849ad7b60f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:46.177185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.179042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.185118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.186476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87be83-f911-4008-a247-478e61996234 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.199266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 949/1880] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:46.201738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.202798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.203166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.227603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:46.229315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.265129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9183310-5838-4ca0-9bbb-68d847422b3a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.271753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.273212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.273500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.273774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] PATH_INFO: `/v3/users/1fd1e37f5f9d449c8fd5277a8fa01cfc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.275058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd1e37f5f9d449c8fd5277a8fa01cfc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.277246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.306291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 950/1881] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:46.316982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.318968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.336875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd1e37f5f9d449c8fd5277a8fa01cfc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.337551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd1e37f5f9d449c8fd5277a8fa01cfc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.346766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd1e37f5f9d449c8fd5277a8fa01cfc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.347477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd1e37f5f9d449c8fd5277a8fa01cfc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.364212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:46.366324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ee3a1234-e357-4b3f-afee-2bb348f2972c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd1e37f5f9d449c8fd5277a8fa01cfc'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.387392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 932/1882] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:46 2022] DELETE /identity/v3/users/1fd1e37f5f9d449c8fd5277a8fa01cfc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:46.392153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.392450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.392735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.392907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dba71b-292c-44a9-a519-8c188b503b8f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.411153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:46.413486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.442530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:46.452991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.453585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.453871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.454334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.456424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.485166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 933/1883] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:46.498981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.501313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.517407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a728-0749-4cd6-9fd7-8a7b36040c6b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.523439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.524890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.525643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.526364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.533143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.534862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.537781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.539493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.548170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 951/1884] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:19:46.549076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.550926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 934/1885] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:46.551482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:46.565615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcad78-21e2-469c-b004-4cec0f9e8bfc service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.570758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.571779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.572630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.572917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] PATH_INFO: `/v3/projects/f8533a97f5c14200ada0a684f7282afa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.574336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8533a97f5c14200ada0a684f7282afa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.576313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.627093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8533a97f5c14200ada0a684f7282afa', 'request_id': 'req-a6852cde-47dd-4b31-a22c-3423ff33ed8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.633856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8533a97f5c14200ada0a684f7282afa', 'request_id': 'req-a6852cde-47dd-4b31-a22c-3423ff33ed8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.640633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8533a97f5c14200ada0a684f7282afa', 'request_id': 'req-a6852cde-47dd-4b31-a22c-3423ff33ed8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.655486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] The token cache is being invalidate because project f8533a97f5c14200ada0a684f7282afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:46.655934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.656931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.659616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 952/1886] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:46 2022] DELETE /identity/v3/projects/f8533a97f5c14200ada0a684f7282afa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:46.664571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:46.675136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56276cba-238c-48b0-95ae-24f1d6d0550e service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:46.740573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:46.741291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:46.741576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:46.741852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] PATH_INFO: `/v3/users/359eba528a824635867677f19be44d75` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:46.743102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] RBAC: Authorizing `identity:delete_user(user_id=359eba528a824635867677f19be44d75)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:46.745088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:46.798800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359eba528a824635867677f19be44d75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.799502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359eba528a824635867677f19be44d75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.807604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359eba528a824635867677f19be44d75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.808377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359eba528a824635867677f19be44d75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.827826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359eba528a824635867677f19be44d75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:46.845635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 935/1887] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:46 2022] DELETE /identity/v3/users/359eba528a824635867677f19be44d75 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.137979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.139475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.154065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6852cde-47dd-4b31-a22c-3423ff33ed8e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.177582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:47.226893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.227816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.228170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.228506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] PATH_INFO: `/v3/projects/3ea146d4a6eb4fe5b485a1b95a6b7b5f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.229764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea146d4a6eb4fe5b485a1b95a6b7b5f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.231797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:47.286502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea146d4a6eb4fe5b485a1b95a6b7b5f', 'request_id': 'req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.293586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea146d4a6eb4fe5b485a1b95a6b7b5f', 'request_id': 'req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.300010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea146d4a6eb4fe5b485a1b95a6b7b5f', 'request_id': 'req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.314216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] The token cache is being invalidate because project 3ea146d4a6eb4fe5b485a1b95a6b7b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:47.314656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.315312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.317103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 953/1888] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/projects/3ea146d4a6eb4fe5b485a1b95a6b7b5f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.321870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.323519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.336866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f2ab3-126a-4f2c-877d-d7b27f70458c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.377413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:47.405716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.406613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.406869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.407139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] PATH_INFO: `/v3/users/0554d1f0ffb249cf94a8c5a10d722451` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.408359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0554d1f0ffb249cf94a8c5a10d722451)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.410372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:47.448709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.453011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.475592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0554d1f0ffb249cf94a8c5a10d722451'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.476308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0554d1f0ffb249cf94a8c5a10d722451'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.486076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0554d1f0ffb249cf94a8c5a10d722451'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.486076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0554d1f0ffb249cf94a8c5a10d722451'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.504333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556b1b3-af08-4ab3-a3f0-76a48ba9fa57 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.509518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.510632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.510992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.511366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] PATH_INFO: `/v3/projects/16fbf99a7a254e1ea781b57898102f18` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.512847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16fbf99a7a254e1ea781b57898102f18)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.515712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:47.529135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0554d1f0ffb249cf94a8c5a10d722451'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.555592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 936/1889] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/users/0554d1f0ffb249cf94a8c5a10d722451 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.555592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.559801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.584491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16fbf99a7a254e1ea781b57898102f18', 'request_id': 'req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.593260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16fbf99a7a254e1ea781b57898102f18', 'request_id': 'req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.606678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16fbf99a7a254e1ea781b57898102f18', 'request_id': 'req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.626993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] The token cache is being invalidate because project 16fbf99a7a254e1ea781b57898102f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:47.627508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.628664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.632243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 954/1890] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/projects/16fbf99a7a254e1ea781b57898102f18 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.637726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.639447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.651822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef1aee-eced-4144-baf3-82896142431f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.696448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:47.723093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.724017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.724481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.724722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] PATH_INFO: `/v3/projects/a75e8735391a43888b95675b3a3e7dd7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.725963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75e8735391a43888b95675b3a3e7dd7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.727830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b428e-4c9a-4c7a-8407-5f7e34a862d0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.728063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:47.732748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.733569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.733850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.734123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] PATH_INFO: `/v3/users/3f1dd5876715440e99be8906aafb8eab` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.735401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1dd5876715440e99be8906aafb8eab)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.737360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:47.747493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:47.793774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1dd5876715440e99be8906aafb8eab'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.797926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75e8735391a43888b95675b3a3e7dd7', 'request_id': 'req-8eedc863-9487-494b-bece-1bc797be9122'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.801037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1dd5876715440e99be8906aafb8eab'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.801739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1dd5876715440e99be8906aafb8eab'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.804547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75e8735391a43888b95675b3a3e7dd7', 'request_id': 'req-8eedc863-9487-494b-bece-1bc797be9122'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.810814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1dd5876715440e99be8906aafb8eab'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.811714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1dd5876715440e99be8906aafb8eab'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.813619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75e8735391a43888b95675b3a3e7dd7', 'request_id': 'req-8eedc863-9487-494b-bece-1bc797be9122'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.828684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] The token cache is being invalidate because project a75e8735391a43888b95675b3a3e7dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:47.829294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.829628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8eedc863-9487-494b-bece-1bc797be9122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:47.831699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 937/1891] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/projects/a75e8735391a43888b95675b3a3e7dd7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.832358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 955/1892] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/users/3f1dd5876715440e99be8906aafb8eab => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:47.836768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db01b0-b0be-48a9-8006-21d53d368ba7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.837058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db01b0-b0be-48a9-8006-21d53d368ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.837456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db01b0-b0be-48a9-8006-21d53d368ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.863497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-86db01b0-b0be-48a9-8006-21d53d368ba7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:47.865490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86db01b0-b0be-48a9-8006-21d53d368ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.871986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:47.873527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:47.968410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 938/1893] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:47.971826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e73d-3991-44b3-982b-7c6d655d6c43 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:47.980701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.981069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.981347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.983406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:47.984522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:47.985086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:47.985529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] PATH_INFO: `/v3/projects/b6e96b59eac744bfbc06f522201cf2d2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:47.986925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e96b59eac744bfbc06f522201cf2d2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:47.989112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.001877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.003118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.011784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 939/1894] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:48.015997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.026389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013c476-dac4-41ac-b0a6-750fdc600e36 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.031682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.032350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.032708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.032981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.033712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:48.034065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.035738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.046949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e96b59eac744bfbc06f522201cf2d2', 'request_id': 'req-ba42ccaa-88c3-47e2-9e57-9714aed5464d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.051997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 940/1895] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:19:48.056992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.057114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e96b59eac744bfbc06f522201cf2d2', 'request_id': 'req-ba42ccaa-88c3-47e2-9e57-9714aed5464d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.058527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:48.068257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e96b59eac744bfbc06f522201cf2d2', 'request_id': 'req-ba42ccaa-88c3-47e2-9e57-9714aed5464d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.069174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa639-bfb8-43de-9472-c21979cddc9f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.087621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] The token cache is being invalidate because project b6e96b59eac744bfbc06f522201cf2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:48.088068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.088818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-ba42ccaa-88c3-47e2-9e57-9714aed5464d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.091285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 956/1896] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:47 2022] DELETE /identity/v3/projects/b6e96b59eac744bfbc06f522201cf2d2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:48.095493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb213-a393-4de9-ab47-e2ce9f3db4fc None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.095782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb213-a393-4de9-ab47-e2ce9f3db4fc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.096061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eb213-a393-4de9-ab47-e2ce9f3db4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.115895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-369eb213-a393-4de9-ab47-e2ce9f3db4fc None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.118047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-369eb213-a393-4de9-ab47-e2ce9f3db4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.137932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.138617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.138903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.139180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.139933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.142148 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.166857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:48.179292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 941/1897] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.181732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.184573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.193576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 957/1898] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:48.204172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.204504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.206304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.236097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.236097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.261629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e3be5-51b6-4236-b9c9-1931924e3ee6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.267599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.268302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.268585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.268856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.269542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.272146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.299500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 942/1899] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.300399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.310941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8ddf-c10a-4d0b-97b7-28af01f34796 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.315782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.316492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.316777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.317043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.317869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.319705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.326333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 958/1900] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:48.328251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.330484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.332356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 943/1901] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:48.333215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.345319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426c2f4-58e9-4fec-887f-4431c1e29f60 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.347951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009597ed-14b9-41d3-91ac-bb47bfb09a37 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.351259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.352024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.352372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.352703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.353374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:48.353676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.354426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.355326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.355439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.355614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.355890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] PATH_INFO: `/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.357221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9ca6dcc2c644baa1ac79628bb7fd1d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.359308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.369696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 944/1902] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:48.370656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.383319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162032c5-17d6-4426-9f9d-d7c52c997bcb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.388133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.388791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.389066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.389334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.390083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.392318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.403740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 945/1903] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:48.404597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.412604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9ca6dcc2c644baa1ac79628bb7fd1d', 'request_id': 'req-3240df96-ac4c-4613-8532-792f3668fbce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.417700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8b9a-2f7d-4c90-bf04-223e217e5a74 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.418487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9ca6dcc2c644baa1ac79628bb7fd1d', 'request_id': 'req-3240df96-ac4c-4613-8532-792f3668fbce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.425102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9ca6dcc2c644baa1ac79628bb7fd1d', 'request_id': 'req-3240df96-ac4c-4613-8532-792f3668fbce'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.437738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] The token cache is being invalidate because project 7a9ca6dcc2c644baa1ac79628bb7fd1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:48.438026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.438653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:48.440269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 959/1904] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:48 2022] DELETE /identity/v3/projects/7a9ca6dcc2c644baa1ac79628bb7fd1d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:48.443821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.443821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.455614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:48.481241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.481824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.482152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.482420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.483051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.485195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdbeec-6c85-4672-acec-718cbbaca8c3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.517452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 946/1905] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.518582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.518871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.519149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.531125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240df96-ac4c-4613-8532-792f3668fbce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.540397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.541625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.586065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:48.608447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.609402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.609692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.609966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] PATH_INFO: `/v3/projects/87f0af6b65fd486da0712e3ac1b7c054/users/c275ededb48a45359639a9f26b64a9e6/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.619632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 947/1906] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:48.620709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.622144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.635670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95106-9cd4-477e-8135-fec79cd3d0a3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.637259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f0af6b65fd486da0712e3ac1b7c054, user_id=c275ededb48a45359639a9f26b64a9e6, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.640597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18c6a7-486e-403c-8244-fb291296ba63 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.641792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.642379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.642747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.643113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.643944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.646657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b17c9d-0984-4e5d-9d2d-a4e3a1cf82c2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.659416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 960/1907] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:48 2022] PUT /identity/v3/projects/87f0af6b65fd486da0712e3ac1b7c054/users/c275ededb48a45359639a9f26b64a9e6/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:48.660822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.661100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.661374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.683006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.684712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.688800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 948/1908] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.688800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.689032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.689115 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.752694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] MFA Rules not processed for user `c275ededb48a45359639a9f26b64a9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:48.754432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:48.790097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 961/1909] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:48.790097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.799580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef1e1c-0801-4eb0-b8cb-f18ca407cb51 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.806839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.807636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.807927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.808319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.809119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.810806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.822339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 962/1910] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:48.822339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.830385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 949/1911] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:48.833916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5427922-16c8-4e84-a2f0-4fc2bc4acdae admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.834902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.838863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.839648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.840001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.840351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.841149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:48.841522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.843203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.846381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632ffe9-8ba0-4611-849d-60d2fe14682c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.852592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.853264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.853547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.853817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.854544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.856574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.860916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 963/1912] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:48.860916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.870055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 950/1913] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:48.870055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.872958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec828-7012-455c-b6aa-8342e4503bf4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.877878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.878299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec7ee0-4ec0-4a40-9099-3838901ed2ff admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.878527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.878841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.879132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.879857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.882465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.886175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.886947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.887234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.887545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.888521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:48.890492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:48.901449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:48.914798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 964/1914] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.915885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.921068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 951/1915] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:48.922046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:48.982328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3189fee-0e6f-4e8e-adcc-6810e5c3324c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.990481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.991559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.991765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956bd48-e814-4bb7-b0ef-b0b5fc5a9bc4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:48.991879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.992104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] PATH_INFO: `/v3/projects/6eb8697e66db42559cbbb01075896b9d/users/9ee1b5d51aba43ce9f1741b0dbb6716b/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.996618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:48.997290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:48.997922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:48.997922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:48.998514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.000576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.018526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb8697e66db42559cbbb01075896b9d, user_id=9ee1b5d51aba43ce9f1741b0dbb6716b, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.022732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.028020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 952/1916] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:48 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:49.028020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.037157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8509eb-c9b8-457f-bdcf-931af87aa7a5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.042249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.042864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.043161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.043469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.044166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.046226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597dbda9-a826-4da3-92ef-e7ec130e02ea admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.047871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 965/1917] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:48 2022] PUT /identity/v3/projects/6eb8697e66db42559cbbb01075896b9d/users/9ee1b5d51aba43ce9f1741b0dbb6716b/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:49.048719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.056833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:49.085683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 953/1918] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/users => generated 282 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:49.087321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.087787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.088216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.125269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] MFA Rules not processed for user `9ee1b5d51aba43ce9f1741b0dbb6716b`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:49.126808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:49.135641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da358c-8519-4188-bbf9-9c81c66d72bd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.142282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.143179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.143490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.143881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.144777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.147148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.161143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 966/1919] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:49.162644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.174051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4b24e-9b12-42a6-84ae-655bd2b3de84 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.179751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.180429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.180711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.181045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.181694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.183457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.199204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 967/1920] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:49.200067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.213495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56344e9-02aa-403a-a04d-ecf43ddce579 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.219643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.220450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.220736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.220999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] PATH_INFO: `/v3/projects/f576fd4e160d4533aabc3901d46571e6/users/7406f6113f5a4ccf8bfda4976a4e917e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.233573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 954/1921] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:49.234251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.242868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f576fd4e160d4533aabc3901d46571e6, user_id=7406f6113f5a4ccf8bfda4976a4e917e, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.246623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246bdc-a3a7-4ae2-803a-9e8f77ca3207 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.247909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.263604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.264370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.264726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.265167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.265911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.267826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.269483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 968/1922] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:49 2022] PUT /identity/v3/projects/f576fd4e160d4533aabc3901d46571e6/users/7406f6113f5a4ccf8bfda4976a4e917e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:49.270328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.272505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:49.281761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 955/1923] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:49.281761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.292185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af9da-5ad3-44c4-a589-ebf05473f62c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.330194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:49.348269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf74a-7b61-4e33-b968-846b0532a731 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.351761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.352439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.352721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.353046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.353484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.353776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.354135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.354421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.354701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.355408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.376038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 956/1924] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:49.376038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.385625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563bb06-04e5-478a-a725-18b00623d2c6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.392001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.392992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.393278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.393577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] PATH_INFO: `/v3/projects/431f0fe7929a4ab29e6944a329361bed/users/e1f00ad533c846b2a8f81a6ca70434a4/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.412635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=431f0fe7929a4ab29e6944a329361bed, user_id=e1f00ad533c846b2a8f81a6ca70434a4, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.415566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.430820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 957/1925] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:49 2022] PUT /identity/v3/projects/431f0fe7929a4ab29e6944a329361bed/users/e1f00ad533c846b2a8f81a6ca70434a4/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:49.431827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.440071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.441647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ed9d0-568e-485c-befd-1f41e41dee1b service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.454742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 969/1926] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:49.456195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.456606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.456891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.492343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:49.495648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] MFA Rules not processed for user `e1f00ad533c846b2a8f81a6ca70434a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:49.499330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:49.525694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b8142-1693-45de-8a3a-40a378e757ab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.531982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.533369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.533369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.533369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] PATH_INFO: `/v3/projects/f576fd4e160d4533aabc3901d46571e6/users/7406f6113f5a4ccf8bfda4976a4e917e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.535218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f576fd4e160d4533aabc3901d46571e6, user_id=7406f6113f5a4ccf8bfda4976a4e917e, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.538241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f750f77-029e-44c5-830b-eecde9c15155 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.551222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 958/1927] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:49 2022] PUT /identity/v3/projects/f576fd4e160d4533aabc3901d46571e6/users/7406f6113f5a4ccf8bfda4976a4e917e/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:49.561391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.561797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.561953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.605357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] MFA Rules not processed for user `7406f6113f5a4ccf8bfda4976a4e917e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:49.607306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:49.610148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 970/1928] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:49.615965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.631311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72fb1-a10e-4ac1-9e6d-a7a1ca4380ef None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.705620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.706488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.706777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.707086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.707849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.710118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.717289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 959/1929] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:49.732287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.737779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:49.746554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 971/1930] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:49.750669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.765453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:49.828796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b001d-aa96-4d99-b637-bda8457d8aad None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.894830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230cdb2-31cb-48a0-8176-a2c03b3d69cc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.900198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.900798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.901078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.901351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.902037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.904026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.910036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.910654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.910939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.911210 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.937474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 972/1931] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:49 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:49.939152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:49.950359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:49.952167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148e270-1a56-4bbb-9255-32375867ee28 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:49.961621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:49.963599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:49.964230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:49.964676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:49.965907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:49.967827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:49.989316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 973/1932] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:49.991440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.005072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb1b3-b0e4-4624-a674-29ddcb292b9b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.011189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.013365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.013657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.013935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] PATH_INFO: `/v3/projects/417b096a8abe4266bb6564184b7c03aa/users/fbff9aad3338401e9f3180c9295f2075/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.028033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.030060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.032962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417b096a8abe4266bb6564184b7c03aa, user_id=fbff9aad3338401e9f3180c9295f2075, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.035860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.058023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 974/1933] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:49 2022] PUT /identity/v3/projects/417b096a8abe4266bb6564184b7c03aa/users/fbff9aad3338401e9f3180c9295f2075/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:50.059646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 960/1934] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:49 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:50.063427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.164317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633edf9-edd7-4ff8-b697-54ba70bea672 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.170234 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.171037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.171200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.171560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.175559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.178193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdffa2-0de0-44c7-9b3d-563b48669353 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.189951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 975/1935] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:50.195649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.210141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43be302-547b-4525-bb6e-5056ab998256 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.216187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.217234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.217471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.217674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] PATH_INFO: `/v3/projects/417b096a8abe4266bb6564184b7c03aa/users/fbff9aad3338401e9f3180c9295f2075/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.220641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=417b096a8abe4266bb6564184b7c03aa, user_id=fbff9aad3338401e9f3180c9295f2075, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.223149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.235928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 961/1936] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:50 2022] PUT /identity/v3/projects/417b096a8abe4266bb6564184b7c03aa/users/fbff9aad3338401e9f3180c9295f2075/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:50.249720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17da46-6187-42a0-a849-85539086c762 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.249720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17da46-6187-42a0-a849-85539086c762 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.249720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17da46-6187-42a0-a849-85539086c762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.258087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.282091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8c17da46-6187-42a0-a849-85539086c762 None None] MFA Rules not processed for user `fbff9aad3338401e9f3180c9295f2075`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:50.283525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8c17da46-6187-42a0-a849-85539086c762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:50.315631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6de3c-70b0-4b5b-855d-f0390c758557 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.319748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.320393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.320679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.320958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.331837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:50.355327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 976/1937] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:50.367382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17da46-6187-42a0-a849-85539086c762 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.378252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17da46-6187-42a0-a849-85539086c762 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.382766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.383442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.383721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.383993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.390827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.392888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.393709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.395133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.401914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 962/1938] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:50 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:50.404645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 977/1939] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:50 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:19:50.479286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:50.488281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a7328-8335-44a1-8729-66e1b7db0e2e service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:50.493070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:50.493716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:50.494001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:50.494280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:50.562932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:50.564509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b590b-40b2-4b72-94f3-25cbbd01dc40 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:50.574554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 963/1940] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:50 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:52.793153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:52.854709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f816ae9-f243-48fd-b2f1-a777fce906c3 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:52.859310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:52.860127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:52.860411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:52.860688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] PATH_INFO: `/v3/users/9ee1b5d51aba43ce9f1741b0dbb6716b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:52.861862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee1b5d51aba43ce9f1741b0dbb6716b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:52.863826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:52.910125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee1b5d51aba43ce9f1741b0dbb6716b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:52.917777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee1b5d51aba43ce9f1741b0dbb6716b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:52.918461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee1b5d51aba43ce9f1741b0dbb6716b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:52.927656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee1b5d51aba43ce9f1741b0dbb6716b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:52.928550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee1b5d51aba43ce9f1741b0dbb6716b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:52.947975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 978/1941] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:52 2022] DELETE /identity/v3/users/9ee1b5d51aba43ce9f1741b0dbb6716b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:52.952465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:52.952943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:52.953055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:52.965086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:52.971767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:52.972979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:53.033086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 964/1942] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:53.042954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.044549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:53.109302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943db90-149b-43f2-bcfd-b361c51f67dc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.116991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.117620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.117949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.118395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.124431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:53.129417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.132028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefe31d-b597-4d84-932e-dbf2b144853a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.142007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 979/1943] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:53.424666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.426082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:53.447809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a416ee-8156-4d1a-b3e6-d241f1c0944a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.453653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.454613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.454915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.455186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] PATH_INFO: `/v3/projects/6eb8697e66db42559cbbb01075896b9d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.456424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb8697e66db42559cbbb01075896b9d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.459057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.518818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb8697e66db42559cbbb01075896b9d', 'request_id': 'req-4823e107-191d-4f59-8168-64924774f57b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:53.526532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb8697e66db42559cbbb01075896b9d', 'request_id': 'req-4823e107-191d-4f59-8168-64924774f57b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:53.537479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb8697e66db42559cbbb01075896b9d', 'request_id': 'req-4823e107-191d-4f59-8168-64924774f57b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:53.548628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] The token cache is being invalidate because project 6eb8697e66db42559cbbb01075896b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:53.548915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:53.549590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4823e107-191d-4f59-8168-64924774f57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:53.551161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 965/1944] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:53 2022] DELETE /identity/v3/projects/6eb8697e66db42559cbbb01075896b9d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:53.556869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.557274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.557569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.575614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:53.576888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:53.639658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 980/1945] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:53.648188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.648547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.648893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.667536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:53.668834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:53.677281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 966/1946] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:53.681431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.691924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13b506-3f79-4f7c-b43e-7772cc72646b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.697151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.697813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.698090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.698411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.699178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:53.699513 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.701146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.715439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 981/1947] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:53.719540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.729684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8e350-c2da-4836-95e7-1b4af7358a4b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.734442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.735095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.735415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.735695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.736389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.738311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.764158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 967/1948] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:53 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:53.769141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.780613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2819b-47e2-47da-a8cc-d1be4f561ea9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.838021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.838765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.838964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.839269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.839952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.841865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.862729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 982/1949] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:53 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:53.867334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.877615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fca4f2-6180-4ac8-8c95-694a261b8a80 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.882956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.883642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.883920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.884189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.884911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.886522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.897824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 968/1950] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:53.901696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.911610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7dc3-1b6d-4d19-8671-f07b656e0cef admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.916235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.916889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.917172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.917452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.918169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.919791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b679e0-3b89-451c-a8b4-1a3dbe904f7c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.929633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 983/1951] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:53.933713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:53.943863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5becb-a848-4a77-b304-8c8c0506452e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:53.948378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:53.949235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.949512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.949780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] PATH_INFO: `/v3/projects/7ee42c07521343108b37b09f4d60b55b/users/b575df475c3a4e0db0a4f4b53995f486/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:53.964843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee42c07521343108b37b09f4d60b55b, user_id=b575df475c3a4e0db0a4f4b53995f486, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:53.967517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:53.980945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 969/1952] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:53 2022] PUT /identity/v3/projects/7ee42c07521343108b37b09f4d60b55b/users/b575df475c3a4e0db0a4f4b53995f486/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:53.992556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:53.992851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:53.993130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:54.023533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] MFA Rules not processed for user `b575df475c3a4e0db0a4f4b53995f486`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:54.024748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:54.072459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:54.091272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 984/1953] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:54.102214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:54.103922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:54.135562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:54.164140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd2a4-7c28-4eb6-a1f7-66d908e8bf02 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:54.167707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:54.168345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:54.168628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:54.168901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:54.177467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:54.178887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552525e-a885-4243-b815-2762b08651e6 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:54.187524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 970/1954] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:55.738418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:55.740416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:55.815790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431e312-fe00-4e55-8c78-1d0e90da232a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:55.820502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:55.821467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:55.821749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:55.822024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] PATH_INFO: `/v3/users/313f40bacca245f79aa0dc4ba710c605` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:55.823217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313f40bacca245f79aa0dc4ba710c605)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:55.825202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:55.912662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313f40bacca245f79aa0dc4ba710c605'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:55.923017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313f40bacca245f79aa0dc4ba710c605'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:55.923792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313f40bacca245f79aa0dc4ba710c605'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:55.936935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313f40bacca245f79aa0dc4ba710c605'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:55.937616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313f40bacca245f79aa0dc4ba710c605'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:55.970947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 985/1955] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:55 2022] DELETE /identity/v3/users/313f40bacca245f79aa0dc4ba710c605 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:55.975218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:55.975672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:55.975824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:56.005741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:56.009706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:56.173218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 971/1956] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:56.188927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:56.188927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:56.298087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978eb87-bbd7-4d81-b6b9-a1d80ca3f3a9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:56.309368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:56.311592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:56.314012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:56.316303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:56.335569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:56.338609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:56.360043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 986/1957] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:56.828457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:56.830094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:56.843300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e04a3-89a5-4059-9e8f-de22fe143db7 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:56.848126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:56.849061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:56.849315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:56.849596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] PATH_INFO: `/v3/projects/315f9bf502054d759f0a2dae89fabe55` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:56.850816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=315f9bf502054d759f0a2dae89fabe55)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:56.852778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:56.907225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315f9bf502054d759f0a2dae89fabe55', 'request_id': 'req-d8477e04-4598-4ac1-bcd8-189886edf576'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:56.914837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315f9bf502054d759f0a2dae89fabe55', 'request_id': 'req-d8477e04-4598-4ac1-bcd8-189886edf576'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:56.923491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315f9bf502054d759f0a2dae89fabe55', 'request_id': 'req-d8477e04-4598-4ac1-bcd8-189886edf576'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:56.935367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] The token cache is being invalidate because project 315f9bf502054d759f0a2dae89fabe55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:56.935837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:56.936621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:56.938503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 972/1958] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:56 2022] DELETE /identity/v3/projects/315f9bf502054d759f0a2dae89fabe55 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:56.942404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:56.952831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8003028-5188-41f5-9793-8736f91a9c5a service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:56.961906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:57.023757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.024649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.025534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.025534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] PATH_INFO: `/v3/users/880bf10d56634f31b37fdf613b73da3e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.026634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=880bf10d56634f31b37fdf613b73da3e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.028597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.076798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880bf10d56634f31b37fdf613b73da3e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.084185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880bf10d56634f31b37fdf613b73da3e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.084850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880bf10d56634f31b37fdf613b73da3e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.092978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880bf10d56634f31b37fdf613b73da3e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.093706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-65a19ca9-151a-4199-be4c-3c95bed6f5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880bf10d56634f31b37fdf613b73da3e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.110849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 987/1959] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:56 2022] DELETE /identity/v3/users/880bf10d56634f31b37fdf613b73da3e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:57.156598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.158217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.173351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477e04-4598-4ac1-bcd8-189886edf576 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.227266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:57.239778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.240995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.241452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.241727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] PATH_INFO: `/v3/users/b575df475c3a4e0db0a4f4b53995f486` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.242936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b575df475c3a4e0db0a4f4b53995f486)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.244942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.297646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b575df475c3a4e0db0a4f4b53995f486'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.298533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b575df475c3a4e0db0a4f4b53995f486'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.308161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b575df475c3a4e0db0a4f4b53995f486'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.309079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b575df475c3a4e0db0a4f4b53995f486'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.325031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b575df475c3a4e0db0a4f4b53995f486'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.340240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 973/1960] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:19:57 2022] DELETE /identity/v3/users/b575df475c3a4e0db0a4f4b53995f486 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:57.349740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.352042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.352603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.372792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:57.374077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.447206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 988/1961] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:57.458785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.460442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.478336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.480038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.499704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b80af-a5f4-46ba-9af8-78f255b278a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.507964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.511524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.512044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.512553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] PATH_INFO: `/v3/projects/dc60a823692049178d3fc60433bee1cb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.515460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc60a823692049178d3fc60433bee1cb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.517847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.538548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f98c1b-41ce-4fc2-a204-1bb2f203b5f5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.544950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.545613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.545892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.546166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.559833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.561363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:57.562891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34076035-b089-4c04-bf23-54964454fc11 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.577003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 974/1962] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:57.582743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc60a823692049178d3fc60433bee1cb', 'request_id': 'req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.589028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc60a823692049178d3fc60433bee1cb', 'request_id': 'req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.595403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc60a823692049178d3fc60433bee1cb', 'request_id': 'req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.609766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] The token cache is being invalidate because project dc60a823692049178d3fc60433bee1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:57.610058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.611159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1fb517a9-e0fc-4c5b-9bb3-5d8a499add58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:57.612976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 989/1963] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:57 2022] DELETE /identity/v3/projects/dc60a823692049178d3fc60433bee1cb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:57.618680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.618967 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.619260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.636928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:57.638155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.717917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 975/1964] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:57.725932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.726402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.726868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.749886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:57.751475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.759764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 990/1965] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:57.764028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.774704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a080-bf71-412f-a9fd-6ccd0c6a2efd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.779609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.780267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.780701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.780968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.781694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:57.782004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.783647 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.796144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 976/1966] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:19:57.800384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.811082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4333-5eaf-40a9-9157-f5006e28d89a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.815576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.816287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.816654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.816973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.818567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.820898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.849357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 991/1967] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:57 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:57.854164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.859794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:57.867645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2aae1-0f33-4860-8536-23eef1af2fae admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.869558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.871524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:57.938841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb958df-8051-4d0d-9313-9765dd89ec08 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:57.943592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.944456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.944736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.945008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] PATH_INFO: `/v3/projects/7ee42c07521343108b37b09f4d60b55b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.945815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:57.946191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee42c07521343108b37b09f4d60b55b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.946518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:57.946802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:57.947076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:57.947810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:57.948147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.949728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:57.971492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 977/1968] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:57 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:57.976062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:57.985944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28acff2-a12f-428c-a443-448fa4f6afc7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.000558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee42c07521343108b37b09f4d60b55b', 'request_id': 'req-effa3e9f-af58-4020-b6df-06ad0649bbee'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:58.006432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee42c07521343108b37b09f4d60b55b', 'request_id': 'req-effa3e9f-af58-4020-b6df-06ad0649bbee'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:58.012618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee42c07521343108b37b09f4d60b55b', 'request_id': 'req-effa3e9f-af58-4020-b6df-06ad0649bbee'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:58.028777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] The token cache is being invalidate because project 7ee42c07521343108b37b09f4d60b55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:19:58.029067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:58.029711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-effa3e9f-af58-4020-b6df-06ad0649bbee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:19:58.031592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 992/1969] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:19:57 2022] DELETE /identity/v3/projects/7ee42c07521343108b37b09f4d60b55b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:58.035926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f034ae-551b-4c64-9021-6c5bc6663ebe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.036855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f034ae-551b-4c64-9021-6c5bc6663ebe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.037283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f034ae-551b-4c64-9021-6c5bc6663ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.050223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.051154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.051982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.052449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.053196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.054826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.062297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-76f034ae-551b-4c64-9021-6c5bc6663ebe None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:58.062297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-76f034ae-551b-4c64-9021-6c5bc6663ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.068016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 978/1970] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:58.069058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.070394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 993/1971] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:58.070687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.077699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.077997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.078276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.083695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096bba-a98a-4501-a29f-73b383358f19 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.088627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.089277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.089561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.089837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.090567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.092204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.096299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:58.097495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.102444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 979/1972] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:58.106448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.106867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 994/1973] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:58.110723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.115411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:58.116749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98dbb0-8806-4a47-a178-b0ae2d6bc662 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.120863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc0e34-855b-44c2-9269-789d15014170 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.122558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.123548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.123835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.124229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] PATH_INFO: `/v3/projects/1ff9d4a0cdd94f82a41685bd2dd1b817/users/98337551eb644af4a6e70eeaa8b5b6a1/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.126995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.127864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.128174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.128671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.129451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:19:58.130554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.132423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.140572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff9d4a0cdd94f82a41685bd2dd1b817, user_id=98337551eb644af4a6e70eeaa8b5b6a1, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.143512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dc6c7-e097-4a87-8eb9-869999cedd1d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.148663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 995/1974] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:19:58.150083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.162256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707acb1-fb28-4577-8ca0-35a4b058669c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.162418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 980/1975] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:58 2022] PUT /identity/v3/projects/1ff9d4a0cdd94f82a41685bd2dd1b817/users/98337551eb644af4a6e70eeaa8b5b6a1/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:58.173591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.173882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.174190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.206683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] MFA Rules not processed for user `98337551eb644af4a6e70eeaa8b5b6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:58.209133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.224384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.225046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.225331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.225720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.226453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.228399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.251324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:58.255000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 996/1976] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:58.259320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.278915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 981/1977] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:58.283305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.294828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8efe0-b05f-4743-8cc8-52b8cf2ac247 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.335569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4718-03e3-4b0d-8376-ddac0876f86c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.340639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.341412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.341694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.342029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.342729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.344736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.369407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.370314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.370731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.371191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.372066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.374179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.390553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 997/1978] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:58.395212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.407181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a66ab-431d-4314-bd77-cb3ff8749ba6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.412132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 982/1979] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:58.412640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.413321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.413607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.414125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.414615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.415101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.416262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.427730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 998/1980] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:58.431721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.441775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09864b-11f9-4db9-be06-6c6aaa007161 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.490036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d77c49-3acf-4104-8a5c-336586a03ced admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.495178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.495998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.496279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.496559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.497245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.499135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:58.500127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.510078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.510761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.511049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.511368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.512103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.513812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.525279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 983/1981] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:19:58.525401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 999/1982] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:19:58.529297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.530590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.548063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b042d-958f-47cb-8a66-48e592265a45 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.548181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f512e86-5ea0-4a7f-a9e9-90dab35486f0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.548263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.548380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.548467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.548467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.548594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.548673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] PATH_INFO: `/v3/projects/c04f22d5bad74880a321f1c0ffe2bd59/users/ff75c0af77e04597a7b5746ccfbdac75/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.548745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.548814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.549581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.551174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.564601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 984/1983] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:58.564771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04f22d5bad74880a321f1c0ffe2bd59, user_id=ff75c0af77e04597a7b5746ccfbdac75, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.565574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.566928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e81519-a57d-4b5b-8fdc-6a6bfb4b973b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.576850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59eb47-8ec5-45d0-8c73-f202c79ca3b5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.581958 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1000/1984] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:58 2022] PUT /identity/v3/projects/c04f22d5bad74880a321f1c0ffe2bd59/users/ff75c0af77e04597a7b5746ccfbdac75/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:58.592066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.592364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.592649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.620239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] MFA Rules not processed for user `ff75c0af77e04597a7b5746ccfbdac75`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:58.621433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.627231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:58.644196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.644966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.645458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.646373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.647203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.648979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.661012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 985/1985] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:58.666205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.677759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702c9fd-f7cb-466f-83a7-ee7372f55c2c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.683002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.683950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.684218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.684526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] PATH_INFO: `/v3/projects/6e2645870e154c17b0cf853a63a3eb06/users/c022fbd41cf04c13972f73b4009bf45e/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.699363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1001/1986] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:19:58.702203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2645870e154c17b0cf853a63a3eb06, user_id=c022fbd41cf04c13972f73b4009bf45e, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.705904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a4653-cef2-4f46-9b13-d2ff167d9b69 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.714254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.715774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.723692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 986/1987] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:58 2022] PUT /identity/v3/projects/6e2645870e154c17b0cf853a63a3eb06/users/c022fbd41cf04c13972f73b4009bf45e/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:58.733629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.733732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.734040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.765017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] MFA Rules not processed for user `c022fbd41cf04c13972f73b4009bf45e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:58.766513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:58.802432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33bf27-9fca-49bb-8964-8223fcf47108 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.851982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 987/1988] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:58.856783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.858051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.858721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.859007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.859393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.888361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:58.920788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae60372-7f3a-4179-903c-0dbebdabcd49 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:58.922324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.923803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.925427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:58.926078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:58.926360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:58.927134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:58.927931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:58.929799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:58.933817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1002/1989] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:19:58.959276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 988/1990] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:58.962401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:58.973376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaeceb1-c834-4ade-b589-63e80a0af55f service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.034443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.035060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.035387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.035669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.036558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.038495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.059036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1003/1991] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:19:58 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:19:59.062822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:59.074875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ea2b8-8fbe-48b2-bf30-c1f47cb79af3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.080126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.080765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.081116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.081461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.082530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.084547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.096211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 989/1992] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:59.100699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:59.111996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2e5c-47c2-4ef7-ab45-f4245a155225 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.117981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.118948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.119353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.119722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] PATH_INFO: `/v3/projects/e8623e27a43142cd82410b8b3257f64d/users/a3c0752fca3f49afab256dc01a6f55a6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.137246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8623e27a43142cd82410b8b3257f64d, user_id=a3c0752fca3f49afab256dc01a6f55a6, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.140066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.154531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1004/1993] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:59 2022] PUT /identity/v3/projects/e8623e27a43142cd82410b8b3257f64d/users/a3c0752fca3f49afab256dc01a6f55a6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:19:59.158606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:59.169961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023a72f-ec9e-47f0-b7d9-8a9afa375bcb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.190017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:19:59.239430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.240112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.240397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.240675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.241501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.243460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669757f-7dc0-438f-95a6-8a8a99a8430f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.261975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 990/1994] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:19:59.263169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:59.276955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac3e17-57a4-4c68-83e1-a6457754c09a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.283043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.283970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.284254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.284583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] PATH_INFO: `/v3/projects/e8623e27a43142cd82410b8b3257f64d/users/a3c0752fca3f49afab256dc01a6f55a6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.287373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8623e27a43142cd82410b8b3257f64d, user_id=a3c0752fca3f49afab256dc01a6f55a6, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.290432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.304876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1005/1995] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:19:59 2022] PUT /identity/v3/projects/e8623e27a43142cd82410b8b3257f64d/users/a3c0752fca3f49afab256dc01a6f55a6/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:19:59.315221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.315596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.316068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.349009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] MFA Rules not processed for user `a3c0752fca3f49afab256dc01a6f55a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:19:59.350535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:59.458890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 991/1996] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:19:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:19:59.522477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:19:59.524381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:19:59.658188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b53ba-33c4-40d5-b4c1-55755a882b92 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:19:59.668058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:19:59.670938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:19:59.671285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:19:59.671547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:19:59.689594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:19:59.691274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:19:59.702472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1006/1997] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:19:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:00.405599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:00.405599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:00.422917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3fd9d-12bb-4cd3-afd6-6a3fffd29803 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:00.429314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:00.429960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:00.430245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:00.430521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:00.493196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:00.494651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5c7f5-9d19-400c-bdf1-857b3277afec service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:00.504535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 992/1998] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:01.404222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:01.404222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:01.435549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:01.496519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80601bc-c794-4c6d-bd9e-80a9c47aa44d service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:01.501850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:01.502959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:01.503470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:01.503949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] PATH_INFO: `/v3/users/c275ededb48a45359639a9f26b64a9e6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:01.506576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c275ededb48a45359639a9f26b64a9e6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:01.508770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:01.586357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c275ededb48a45359639a9f26b64a9e6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:01.598350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c275ededb48a45359639a9f26b64a9e6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:01.599586 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c275ededb48a45359639a9f26b64a9e6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:01.624113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c275ededb48a45359639a9f26b64a9e6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:01.625178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c275ededb48a45359639a9f26b64a9e6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:01.670213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1007/1999] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:01 2022] DELETE /identity/v3/users/c275ededb48a45359639a9f26b64a9e6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:01.671954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:01.672475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:01.672562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:01.690493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:01.691861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:01.735266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:01.760508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 993/2000] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:01.770497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:01.771941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:01.839690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9274a-7890-44fd-8ca2-6059c2831bcd admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:01.844250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:01.844999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:01.845355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:01.845706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:01.857442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:01.859431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:01.872685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1008/2001] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:02.223007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:02.226006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:02.246369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8e138-1702-4b0a-b5c7-0c00cb278874 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:02.274189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:02.275538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:02.275872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:02.280963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] PATH_INFO: `/v3/projects/87f0af6b65fd486da0712e3ac1b7c054` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:02.283959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f0af6b65fd486da0712e3ac1b7c054)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:02.289281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:02.357586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f0af6b65fd486da0712e3ac1b7c054', 'request_id': 'req-c5411a85-e7f7-4695-b0d6-907c45f9c718'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.368475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f0af6b65fd486da0712e3ac1b7c054', 'request_id': 'req-c5411a85-e7f7-4695-b0d6-907c45f9c718'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.385220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f0af6b65fd486da0712e3ac1b7c054', 'request_id': 'req-c5411a85-e7f7-4695-b0d6-907c45f9c718'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.407986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] The token cache is being invalidate because project 87f0af6b65fd486da0712e3ac1b7c054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:02.408331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.412481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.414677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 994/2002] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:02 2022] DELETE /identity/v3/projects/87f0af6b65fd486da0712e3ac1b7c054 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:02.419129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:02.434755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d1bf-3f49-4f58-afbb-e668263dbfea service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:02.499649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:02.520637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:02.521542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:02.521833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:02.522113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] PATH_INFO: `/v3/users/7406f6113f5a4ccf8bfda4976a4e917e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:02.523348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7406f6113f5a4ccf8bfda4976a4e917e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:02.525292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:02.590872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7406f6113f5a4ccf8bfda4976a4e917e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.600557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7406f6113f5a4ccf8bfda4976a4e917e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.601309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7406f6113f5a4ccf8bfda4976a4e917e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.610329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7406f6113f5a4ccf8bfda4976a4e917e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.623435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7406f6113f5a4ccf8bfda4976a4e917e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:02.654357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1009/2003] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:02 2022] DELETE /identity/v3/users/7406f6113f5a4ccf8bfda4976a4e917e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:02.978933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:02.980459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:02.993963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5411a85-e7f7-4695-b0d6-907c45f9c718 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.056889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:03.058876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:03.073562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.074446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.075325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.075609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] PATH_INFO: `/v3/users/e1f00ad533c846b2a8f81a6ca70434a4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.076958 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1f00ad533c846b2a8f81a6ca70434a4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.079282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.165101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1f00ad533c846b2a8f81a6ca70434a4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.166021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1f00ad533c846b2a8f81a6ca70434a4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.195179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1f00ad533c846b2a8f81a6ca70434a4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.196153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1f00ad533c846b2a8f81a6ca70434a4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.200676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348d662-9073-468a-8ecc-c53dacb6df09 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.206468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:03.208818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.210023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.210566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.211055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] PATH_INFO: `/v3/projects/f576fd4e160d4533aabc3901d46571e6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.212721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f576fd4e160d4533aabc3901d46571e6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.213147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-54e57675-28ba-4d79-8824-3c8e38cd2022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1f00ad533c846b2a8f81a6ca70434a4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.215515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.240014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 995/2004] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:02 2022] DELETE /identity/v3/users/e1f00ad533c846b2a8f81a6ca70434a4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:03.246063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.246338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.246631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.280508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:03.282192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:03.287701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f576fd4e160d4533aabc3901d46571e6', 'request_id': 'req-8ee102fa-9ec8-47e5-8149-d9e0fb842005'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.294370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f576fd4e160d4533aabc3901d46571e6', 'request_id': 'req-8ee102fa-9ec8-47e5-8149-d9e0fb842005'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.309345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f576fd4e160d4533aabc3901d46571e6', 'request_id': 'req-8ee102fa-9ec8-47e5-8149-d9e0fb842005'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.311866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:03.348244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] The token cache is being invalidate because project f576fd4e160d4533aabc3901d46571e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:03.348244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.355284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8ee102fa-9ec8-47e5-8149-d9e0fb842005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:03.360904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1010/2005] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:03 2022] DELETE /identity/v3/projects/f576fd4e160d4533aabc3901d46571e6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:03.367018 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40c653-b585-49be-a546-588ad410f819 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.367414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40c653-b585-49be-a546-588ad410f819 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.367696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40c653-b585-49be-a546-588ad410f819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.407808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ca40c653-b585-49be-a546-588ad410f819 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:03.409129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ca40c653-b585-49be-a546-588ad410f819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:03.412784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 996/2006] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:03.423569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:03.424652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1011/2007] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:03.426153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:03.432936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.433273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.433490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.481970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:03.484868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:03.509646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1012/2008] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:03.511939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:03.530413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2ad3b-2940-4d45-9e8b-e34572f9b9a8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.547465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.548409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.551088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.552465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.557773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad536b-20e4-4771-af93-99c3f3bb9484 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.572556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.574618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.576229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.576522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.577583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:03.577904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.578828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.582092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.583635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.610323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 997/2009] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:03.621562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1013/2010] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:03.623073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:03.637554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8bf7d-3ab7-4104-9999-dcd41a0fa8a4 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.646227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.647014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.647528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.647890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.648687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.653001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.713499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 998/2011] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:03 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:03.717738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:03.733011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406b0a1-a0f5-45ed-9ab2-eee6b44ae792 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:03.953653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:03.954628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:03.954917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:03.955199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:03.955973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:03.958710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:03.994867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1014/2012] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:03 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:03.999129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.013644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957d123-9585-4a72-882a-eb2c5433d611 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.057965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.058668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.058955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.059248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.066907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.073632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.107985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 999/2013] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:04.109573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.126827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8d692-7f41-4766-9ed8-2eb3f59808f1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.134401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.135560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.136778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.137484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.138433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.140454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae9b7f-f921-41de-bc5b-7784a4055560 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.156126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1015/2014] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:04.159970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.180351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94e670-7250-40bb-b228-f41ab3053bca admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.197668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.199012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.199504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.199960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] PATH_INFO: `/v3/projects/d3eedab150b140e98fb6c1bb919a9818/users/e78aaac80da04ec79599001101a0738f/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.231939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3eedab150b140e98fb6c1bb919a9818, user_id=e78aaac80da04ec79599001101a0738f, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.235747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.258685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1000/2015] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:04 2022] PUT /identity/v3/projects/d3eedab150b140e98fb6c1bb919a9818/users/e78aaac80da04ec79599001101a0738f/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:04.271804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.273263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.273753 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.331205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] MFA Rules not processed for user `e78aaac80da04ec79599001101a0738f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:04.333055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:04.469890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1016/2016] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:04.474603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.539218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:04.551518 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.553093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:04.618987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898e5a-d23a-4365-aa68-78f19d96505c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.624894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.625676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.625977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.626252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.626959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.628955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.631111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:04.670819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1001/2017] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:04 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:04.671867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.696141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b4b88-0227-4169-8066-5c09bdb679cb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.777820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04ee72-464d-49bf-8203-3e53ac8cc7cd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.784244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.787024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.787335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.787622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] PATH_INFO: `/v3/projects/431f0fe7929a4ab29e6944a329361bed` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.790913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=431f0fe7929a4ab29e6944a329361bed)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.792342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.793174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.793459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.793730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.794436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.798349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.799585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.830924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1002/2018] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:04 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:04.834926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.850003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538ed4-475d-4315-8678-5cea0a910cdd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.868529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:04.869588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:04.869948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:04.870309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:04.882765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431f0fe7929a4ab29e6944a329361bed', 'request_id': 'req-790a360e-578e-4156-8cbe-8d9f11f66982'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:04.884688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:04.886779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:04.895196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431f0fe7929a4ab29e6944a329361bed', 'request_id': 'req-790a360e-578e-4156-8cbe-8d9f11f66982'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:04.904434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431f0fe7929a4ab29e6944a329361bed', 'request_id': 'req-790a360e-578e-4156-8cbe-8d9f11f66982'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:04.911388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1003/2019] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:04.915999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.930017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e930835-4b2a-40dd-aa9a-b1de67b70d79 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:04.933947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] The token cache is being invalidate because project 431f0fe7929a4ab29e6944a329361bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:04.935453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:04.936544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:04.939458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1017/2020] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:04 2022] DELETE /identity/v3/projects/431f0fe7929a4ab29e6944a329361bed => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:04.945687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:04.960692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.065820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.066810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.067753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.068130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] PATH_INFO: `/v3/projects/087883dee46d4c949e94c53701d3153a/users/8a8cf8388951407492fdab11dd3aa033/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.087151 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a360e-578e-4156-8cbe-8d9f11f66982 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.095459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.096684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.097065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.097453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] PATH_INFO: `/v3/users/fbff9aad3338401e9f3180c9295f2075` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.099882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbff9aad3338401e9f3180c9295f2075)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.101916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087883dee46d4c949e94c53701d3153a, user_id=8a8cf8388951407492fdab11dd3aa033, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.102282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.104689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.119232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:05.122891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1004/2021] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:04 2022] PUT /identity/v3/projects/087883dee46d4c949e94c53701d3153a/users/8a8cf8388951407492fdab11dd3aa033/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:05.126906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:05.130504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.165175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:05.172621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbff9aad3338401e9f3180c9295f2075'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.181535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbff9aad3338401e9f3180c9295f2075'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.182710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbff9aad3338401e9f3180c9295f2075'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.193191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbff9aad3338401e9f3180c9295f2075'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.194283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbff9aad3338401e9f3180c9295f2075'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.227905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5619-0e98-494e-985b-f551195e5bd8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.231941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1018/2022] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:04 2022] DELETE /identity/v3/users/fbff9aad3338401e9f3180c9295f2075 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:05.232907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:05.234259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.240656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.241354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.241641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.241969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.242970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.251477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.277110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1005/2023] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:05.282850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:05.285081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.307463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d61ea-60f3-4531-9422-c0648747abd1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.337251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74660ae-db83-4cd1-ad16-ef0946f0a903 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.341290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.342086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.342593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.342872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.345706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.432996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.433912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.434230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.434509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] PATH_INFO: `/v3/projects/087883dee46d4c949e94c53701d3153a/users/8a8cf8388951407492fdab11dd3aa033/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.452443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087883dee46d4c949e94c53701d3153a, user_id=8a8cf8388951407492fdab11dd3aa033, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.458209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d19284-2a06-498f-b52d-2fd8886d4354 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.461963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.464267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.480241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1006/2024] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:05 2022] PUT /identity/v3/projects/087883dee46d4c949e94c53701d3153a/users/8a8cf8388951407492fdab11dd3aa033/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:05.490849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.491189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.491417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.494897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1019/2025] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:05.540356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] MFA Rules not processed for user `8a8cf8388951407492fdab11dd3aa033`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:05.541558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.681899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1007/2026] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:05.710132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:05.712695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.748667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f54995-e7b7-40c0-b7ea-47b732add987 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.789255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:05.791010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:05.824572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:05.846797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:05.848521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.849496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.849796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.850093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] PATH_INFO: `/v3/projects/417b096a8abe4266bb6564184b7c03aa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.851693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=417b096a8abe4266bb6564184b7c03aa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.853920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.882378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d4b6-0a44-4a83-95b1-4533cea3fc98 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:05.886354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:05.887014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:05.887336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:05.887655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:05.898349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:05.900179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c76d0d-943e-493c-a011-6a4e6bb61806 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:05.916715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1008/2027] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:05.947336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '417b096a8abe4266bb6564184b7c03aa', 'request_id': 'req-a267f3f2-da0f-4128-a594-7eaf4c1fe765'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.956013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '417b096a8abe4266bb6564184b7c03aa', 'request_id': 'req-a267f3f2-da0f-4128-a594-7eaf4c1fe765'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:05.965754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '417b096a8abe4266bb6564184b7c03aa', 'request_id': 'req-a267f3f2-da0f-4128-a594-7eaf4c1fe765'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.004676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] The token cache is being invalidate because project 417b096a8abe4266bb6564184b7c03aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:06.004973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.007683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.010716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1020/2028] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:05 2022] DELETE /identity/v3/projects/417b096a8abe4266bb6564184b7c03aa => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:06.016885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdfed-cb78-4846-be66-33605ca43d42 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.017221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdfed-cb78-4846-be66-33605ca43d42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.017498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdfed-cb78-4846-be66-33605ca43d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.042163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6b2cdfed-cb78-4846-be66-33605ca43d42 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:06.043393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6b2cdfed-cb78-4846-be66-33605ca43d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.073051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.075317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.123791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1009/2029] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:06.132321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.132714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.133060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.157761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:06.159819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.163264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267f3f2-da0f-4128-a594-7eaf4c1fe765 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.169708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.170390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.170680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.171437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.175000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.184046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1010/2030] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:06.185297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.205796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e9ea0-72a4-4c71-a92b-a7147c883f3b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.218431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.218646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.218982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.219358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.220853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:06.221258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.224374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.251278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1011/2031] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:06.255084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.278201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2e81-5776-4708-b3af-c959cf11fed4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.291274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.292045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.292342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.292560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.293255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.295159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.299989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.302692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ff5fa-72e4-417d-8395-98c9a8580b83 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.320731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1021/2032] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:06.331272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b4f5-0764-4eda-b3b4-a048b6f9cf5e None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.331272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b4f5-0764-4eda-b3b4-a048b6f9cf5e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.331272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070b4f5-0764-4eda-b3b4-a048b6f9cf5e None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.333492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1022/2033] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:20:06.357207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.357207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.357207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.376615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1012/2034] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:06.377911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.407489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:06.407489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.465625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:06.518245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd032e72-9548-48c5-aaf7-72b7e09b42f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.529066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.530408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.531476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.536089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.536739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.547848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.576596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1023/2035] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:06.591263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.592737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1013/2036] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:06.592955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.599650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.608495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3307-763d-489f-a141-97b305355435 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.613730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.615273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.616289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.617951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.617951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.619449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.628987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a4d70-af4f-461c-b8f7-ddc233022247 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.634946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1014/2037] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:06.636008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.647387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.647586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e11071-a39f-4e3c-ac7a-e8f6473bb5f6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.648067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.648347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.648765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.654054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.654202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.654554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.654904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.655783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.658306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.665478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.665478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a412732-6411-49eb-9245-8ce8ab6079c7 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.675174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1015/2038] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:06.680212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.688394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:06.692685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1024/2039] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:06.698681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817ec6b-353a-48a6-ac5b-80a4dd6a613e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.720509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.723853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.724146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.724738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] PATH_INFO: `/v3/projects/c10270d360324ed2974748fbae02affd/users/18787fd1d17e4ae69e911d895342f746/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.751194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c10270d360324ed2974748fbae02affd, user_id=18787fd1d17e4ae69e911d895342f746, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.753645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.771965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1016/2040] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:06 2022] PUT /identity/v3/projects/c10270d360324ed2974748fbae02affd/users/18787fd1d17e4ae69e911d895342f746/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:06.789515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.789989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.790777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.823968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] MFA Rules not processed for user `18787fd1d17e4ae69e911d895342f746`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:06.825185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.833283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.836467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:06.896598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1025/2041] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:06.902076 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.912272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1eaf-d359-441d-893f-331f1a4670e4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.913732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a56a1-67d6-4f53-800c-69204976b1fe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:06.916862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.917823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.917823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.918075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.918581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:06.918730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.919556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:06.919835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:06.920106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] PATH_INFO: `/v3/users/e78aaac80da04ec79599001101a0738f` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:06.920790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.921311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78aaac80da04ec79599001101a0738f)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:06.923287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:06.949798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1026/2042] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:06.954552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:06.987110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78aaac80da04ec79599001101a0738f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.987835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78aaac80da04ec79599001101a0738f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.997209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78aaac80da04ec79599001101a0738f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:06.997916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78aaac80da04ec79599001101a0738f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:07.027261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27715486-103e-4c57-9d59-a329e9ac6ca1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.035385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f1909dc5-192c-42c8-80df-d4c52945f0ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78aaac80da04ec79599001101a0738f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:07.038284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.039338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.039474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.039774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.040559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.042867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.051314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1017/2043] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:06 2022] DELETE /identity/v3/users/e78aaac80da04ec79599001101a0738f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:07.056498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.056731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.057187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.068015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1027/2044] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:06 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:07.071484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.073216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.079608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:07.080824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.082516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:07.086665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dfe11-d781-4c52-80ce-b70b0ba61f9c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.148306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:07.163515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.164254 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.164359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1018/2045] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:07.169550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.170206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.170459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.172365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.180822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.182973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.191432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1028/2046] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:07.193089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.203688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a9ac-4695-4c0a-a3dc-f92bfa154622 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.208710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.209598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.209881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.210155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] PATH_INFO: `/v3/projects/8eb1a38386fa4e21bae952472a3ee470/users/9965cc3fc6a8428190553b638e327885/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.231662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb1a38386fa4e21bae952472a3ee470, user_id=9965cc3fc6a8428190553b638e327885, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.236724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.252768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1029/2047] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:07 2022] PUT /identity/v3/projects/8eb1a38386fa4e21bae952472a3ee470/users/9965cc3fc6a8428190553b638e327885/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:07.257742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.260561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0163f-8163-44cf-b947-c760313095aa None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.324454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.324663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8273351-7e98-4372-9e98-20bc5dfd99a9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.325610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.326076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.326353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.336701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.337088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.337386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.337671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.337949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.338815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.338926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.341035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b31ce-c32c-4ebb-9b97-1249f947f33c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.350858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1019/2048] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:07.356135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1030/2049] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:07.358547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.371758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef59150-cee3-4e54-804c-b8038cbf97e1 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.376901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.378358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.378646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.378925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] PATH_INFO: `/v3/projects/8eb1a38386fa4e21bae952472a3ee470/users/9965cc3fc6a8428190553b638e327885/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.381831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb1a38386fa4e21bae952472a3ee470, user_id=9965cc3fc6a8428190553b638e327885, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.384915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.403891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1020/2050] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:07 2022] PUT /identity/v3/projects/8eb1a38386fa4e21bae952472a3ee470/users/9965cc3fc6a8428190553b638e327885/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:07.415727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.416112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.416432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.453854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] MFA Rules not processed for user `9965cc3fc6a8428190553b638e327885`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:07.455814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.550794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:07.557352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1031/2051] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:07.568518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.570281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.646841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed12eb-9fed-4e8f-a677-106d1120e32c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.650798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.651494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.651781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.652200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.654489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.730573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.732204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.736188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:07.743320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1021/2052] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:07.814999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:07.817846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:07.922185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a208c-b08a-4fb2-9cd1-2bef140c96ce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:07.928262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:07.929696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:07.929985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:07.930262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] PATH_INFO: `/v3/projects/d3eedab150b140e98fb6c1bb919a9818` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:07.931630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3eedab150b140e98fb6c1bb919a9818)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:07.933456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:07.991349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3eedab150b140e98fb6c1bb919a9818', 'request_id': 'req-b27dbd4c-ea7c-4157-bf32-564933073cf7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:07.998368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3eedab150b140e98fb6c1bb919a9818', 'request_id': 'req-b27dbd4c-ea7c-4157-bf32-564933073cf7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.006891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3eedab150b140e98fb6c1bb919a9818', 'request_id': 'req-b27dbd4c-ea7c-4157-bf32-564933073cf7'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.023172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] The token cache is being invalidate because project d3eedab150b140e98fb6c1bb919a9818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:08.023478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.024153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.025994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1032/2053] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:07 2022] DELETE /identity/v3/projects/d3eedab150b140e98fb6c1bb919a9818 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:08.030676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:08.042901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8b192-77d5-4602-92dd-90bec1aba13f service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:08.116045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:08.116760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.117158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.117533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] PATH_INFO: `/v3/users/8a8cf8388951407492fdab11dd3aa033` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.119001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8cf8388951407492fdab11dd3aa033)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:08.121229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:08.182408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8cf8388951407492fdab11dd3aa033'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.183517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8cf8388951407492fdab11dd3aa033'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.190719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8cf8388951407492fdab11dd3aa033'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.191386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8cf8388951407492fdab11dd3aa033'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.204710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-93e4a1a5-be0d-4dbc-8f4b-f1bcb763ac9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8cf8388951407492fdab11dd3aa033'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.222073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1022/2054] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:08 2022] DELETE /identity/v3/users/8a8cf8388951407492fdab11dd3aa033 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:08.503679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:08.505791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:08.526873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dbd4c-ea7c-4157-bf32-564933073cf7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:08.585186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:08.586065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.586346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.586619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] PATH_INFO: `/v3/projects/087883dee46d4c949e94c53701d3153a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.587838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=087883dee46d4c949e94c53701d3153a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:08.589777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:08.624045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:08.638192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087883dee46d4c949e94c53701d3153a', 'request_id': 'req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.643787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087883dee46d4c949e94c53701d3153a', 'request_id': 'req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.650377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087883dee46d4c949e94c53701d3153a', 'request_id': 'req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.661992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] The token cache is being invalidate because project 087883dee46d4c949e94c53701d3153a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:08.662283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.662891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-bb37d9f0-2e86-47a7-ab65-ccbbaf93b8ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:08.664434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1033/2055] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:08 2022] DELETE /identity/v3/projects/087883dee46d4c949e94c53701d3153a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:08.669289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.669656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.669946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.687926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:08.689139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:08.749072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1023/2056] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:08.757868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.758175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.758466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.777256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:08.778479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:08.786945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1034/2057] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:08.791201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:08.800867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a048a25-751e-424c-b547-68a2cf7bd396 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:08.806065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:08.806718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.807083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.807507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.808249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:08.808603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:08.810384 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:08.822393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1024/2058] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:08.826544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:08.836710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd04a5a-c135-4a15-8639-2653a15025a2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:08.841576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:08.842317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.842616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.842955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.843748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:08.845663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:08.886628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1035/2059] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:08 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:08.888631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:08.898934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:08.906569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceac40e-eee2-47a3-9bc7-18a970f930dd admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:08.981852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:08.982524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:08.982805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:08.983076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:08.984231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:08.985729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.008777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1025/2060] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:08 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:09.013035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.025454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f872-2ddc-4e03-9ce4-0acbec1487a2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.031060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.031925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.032373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.032524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.033248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.034963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.045896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1036/2061] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:09.049931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.063076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406647e-c536-4d10-8d7f-ce1743b02b18 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.068800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.069470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.069752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.070026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.070753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.072400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11b3d3-1b2d-4d21-aa4f-63eea1bcb244 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.084157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1026/2062] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:09.088291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.102574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d607159-c109-4137-b178-fadaca4295dc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.108042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.108931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.109213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.109486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] PATH_INFO: `/v3/projects/b0df507a28e948cb97d5849d4065560e/users/d4918a5aa8da4d979a416f1139be3423/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.125139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0df507a28e948cb97d5849d4065560e, user_id=d4918a5aa8da4d979a416f1139be3423, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.127879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.141619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1037/2063] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:09 2022] PUT /identity/v3/projects/b0df507a28e948cb97d5849d4065560e/users/d4918a5aa8da4d979a416f1139be3423/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:09.152787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.153074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.153387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.187567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] MFA Rules not processed for user `d4918a5aa8da4d979a416f1139be3423`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:09.188790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:09.269090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1027/2064] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:09.280056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.282330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:09.344354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049784-c8c4-42ba-8bfd-8c2ae5883ccf admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.348861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.349460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.349999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.350280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.359959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.361516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9733c3-a647-4410-8b6f-3e797cbeb996 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.373654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1038/2065] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:09.399407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.499427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f8e69-d181-481a-ab18-d443cdb4f5a6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.507214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.509086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.509086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.509086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] PATH_INFO: `/v3/users/d4918a5aa8da4d979a416f1139be3423` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.510943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4918a5aa8da4d979a416f1139be3423)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.520653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.547857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:09.593137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4918a5aa8da4d979a416f1139be3423'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:09.593810 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4918a5aa8da4d979a416f1139be3423'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:09.601874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4918a5aa8da4d979a416f1139be3423'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:09.602576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4918a5aa8da4d979a416f1139be3423'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:09.618575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4918a5aa8da4d979a416f1139be3423'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:09.634965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1028/2066] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:09 2022] DELETE /identity/v3/users/d4918a5aa8da4d979a416f1139be3423 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:09.639524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.639811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.640087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.660834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:09.662411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:09.713873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:09.724026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1039/2067] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:09.733132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:09.734537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:09.799184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1ced-88af-4035-a4ee-3bb366e45557 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:09.803083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:09.803754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:09.804038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:09.804308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:09.813430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:09.814859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c1599-b4ea-46cf-aef6-0445ce4b31a2 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:09.824804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1029/2068] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:10.269621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.272816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.292230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065ca13-af8c-4546-9d9b-6c131e52305a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.297994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.299112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.299469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.300019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] PATH_INFO: `/v3/projects/b0df507a28e948cb97d5849d4065560e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.301555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0df507a28e948cb97d5849d4065560e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.303895 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.354624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0df507a28e948cb97d5849d4065560e', 'request_id': 'req-74a5f965-6c35-4224-a089-1f3458e3ebe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.360132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0df507a28e948cb97d5849d4065560e', 'request_id': 'req-74a5f965-6c35-4224-a089-1f3458e3ebe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.367808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0df507a28e948cb97d5849d4065560e', 'request_id': 'req-74a5f965-6c35-4224-a089-1f3458e3ebe6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.384353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] The token cache is being invalidate because project b0df507a28e948cb97d5849d4065560e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:10.384646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.385424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.386990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1040/2069] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:10 2022] DELETE /identity/v3/projects/b0df507a28e948cb97d5849d4065560e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:10.392098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed939-67c4-46bc-8796-79f8c9d0baeb None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.392517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed939-67c4-46bc-8796-79f8c9d0baeb None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.392659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ed939-67c4-46bc-8796-79f8c9d0baeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.410978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-d65ed939-67c4-46bc-8796-79f8c9d0baeb None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:10.412477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d65ed939-67c4-46bc-8796-79f8c9d0baeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.428973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.430818 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.474000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1030/2070] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:10.481786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.482076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.482354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.497573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5f965-6c35-4224-a089-1f3458e3ebe6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.500114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:10.501310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.502243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.503106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.503490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.503766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] PATH_INFO: `/v3/users/98337551eb644af4a6e70eeaa8b5b6a1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.505469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] RBAC: Authorizing `identity:delete_user(user_id=98337551eb644af4a6e70eeaa8b5b6a1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.507456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.510002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1031/2071] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:10.514032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.523869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f48bf-42ab-4512-af00-54e396f7d940 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.528517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.529167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.529449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.529718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.530435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:10.530734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.532381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.549718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1032/2072] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:10.550706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.562056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c81c8-83a2-4e54-9f16-36b72cfa1309 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.564167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98337551eb644af4a6e70eeaa8b5b6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.567887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.568607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.568957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.569303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.570040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.572078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.573557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98337551eb644af4a6e70eeaa8b5b6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.574273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98337551eb644af4a6e70eeaa8b5b6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.581665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98337551eb644af4a6e70eeaa8b5b6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.582319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1f376b15-a1ac-4f53-8d47-6632dcea331a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98337551eb644af4a6e70eeaa8b5b6a1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:10.588149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:10.599428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:10.604801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1041/2073] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:10 2022] DELETE /identity/v3/users/98337551eb644af4a6e70eeaa8b5b6a1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:10.609956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1033/2074] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:10.610099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.610099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.610099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.610337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.617060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.632489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:10.633192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.687822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6936-18bc-44cc-8ded-bd928a423812 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.694130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1042/2075] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:10.704158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.705483 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:10.747297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.747966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.748250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.748522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.749203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.751125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.762762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a1d8b-5a15-4639-8b69-801ce4c75609 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.766706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.767375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.767664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.767924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.772413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1034/2076] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:10.776529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.778387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.779868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.786235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1509958-9c04-4471-abe2-46d2a6c00a2c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.789468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1043/2077] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:10.790944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.791637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.791924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.792198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.792923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.794522 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.807816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1035/2078] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:10.808949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.819534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda69f58-14e0-488e-91ce-8475267e00f7 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.824134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.824788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.825067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.825339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.826149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.827689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259c190-0f59-491e-892e-2a1d87d4336c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.838725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1044/2079] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:10.842985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:10.854431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15087c2-6479-4013-be6e-1f28504b9a30 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:10.859650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:10.860594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.860866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.861133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] PATH_INFO: `/v3/projects/6706473c227a48888ef1e459916def43/users/bb2f249eee2b4b36a899d0ec0cc6cce4/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.876955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6706473c227a48888ef1e459916def43, user_id=bb2f249eee2b4b36a899d0ec0cc6cce4, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:10.879781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:10.893094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1036/2080] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:10 2022] PUT /identity/v3/projects/6706473c227a48888ef1e459916def43/users/bb2f249eee2b4b36a899d0ec0cc6cce4/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:10.904832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:10.905489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:10.905607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:10.934199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] MFA Rules not processed for user `bb2f249eee2b4b36a899d0ec0cc6cce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:10.935523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.005290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1045/2081] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:11.016772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.018617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.060872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.062847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.080853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab0e08-12ff-41d3-957d-f847f3d8ddc9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.085732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.086505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.086774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.087408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.095738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:11.096118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.097583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.102207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:11.108308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1037/2082] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:11.134678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.138351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4dc6-16df-47a2-b14f-299d1ca5a921 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.144360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.145427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.146067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.146417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] PATH_INFO: `/v3/projects/1ff9d4a0cdd94f82a41685bd2dd1b817` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.147587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b688a9-fc1d-4be1-b7a8-36758b4775b8 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.147876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff9d4a0cdd94f82a41685bd2dd1b817)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.149983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.153860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.155052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.155706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.155993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] PATH_INFO: `/v3/users/bb2f249eee2b4b36a899d0ec0cc6cce4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.159436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2f249eee2b4b36a899d0ec0cc6cce4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.161947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.199982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff9d4a0cdd94f82a41685bd2dd1b817', 'request_id': 'req-9514c752-2199-42ac-afd2-05c2171802cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.205540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff9d4a0cdd94f82a41685bd2dd1b817', 'request_id': 'req-9514c752-2199-42ac-afd2-05c2171802cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.213011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2f249eee2b4b36a899d0ec0cc6cce4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.213155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff9d4a0cdd94f82a41685bd2dd1b817', 'request_id': 'req-9514c752-2199-42ac-afd2-05c2171802cb'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.213692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2f249eee2b4b36a899d0ec0cc6cce4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.221857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2f249eee2b4b36a899d0ec0cc6cce4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.222527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2f249eee2b4b36a899d0ec0cc6cce4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.227088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] The token cache is being invalidate because project 1ff9d4a0cdd94f82a41685bd2dd1b817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:11.227418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.228074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.229659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1046/2083] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:11 2022] DELETE /identity/v3/projects/1ff9d4a0cdd94f82a41685bd2dd1b817 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:11.233476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.234812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.235192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7f4da8b-3335-4c19-adff-81755c8da5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2f249eee2b4b36a899d0ec0cc6cce4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.249397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1038/2084] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:11 2022] DELETE /identity/v3/users/bb2f249eee2b4b36a899d0ec0cc6cce4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:11.254112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.255058 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.255376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.274458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:11.275719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.310616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c752-2199-42ac-afd2-05c2171802cb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.344527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1039/2085] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:11.356726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.358989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.400647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.401548 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.401842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.402325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] PATH_INFO: `/v3/users/c022fbd41cf04c13972f73b4009bf45e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.403615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c022fbd41cf04c13972f73b4009bf45e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.405592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.407477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:11.430975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819f441-f889-4319-bd5b-f3c09261345c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.434622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.435313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.435598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.435873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.446572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.448095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.459410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1040/2086] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:11.469143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c022fbd41cf04c13972f73b4009bf45e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.476102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c022fbd41cf04c13972f73b4009bf45e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.476804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c022fbd41cf04c13972f73b4009bf45e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.484428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c022fbd41cf04c13972f73b4009bf45e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.485133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c022fbd41cf04c13972f73b4009bf45e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.503354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1047/2087] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:11 2022] DELETE /identity/v3/users/c022fbd41cf04c13972f73b4009bf45e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:11.790735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.792382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.818765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86e5e-9f7c-4210-a301-10e0de02f338 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.831801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:11.832729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:11.833472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:11.890238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.891372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.891661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.891939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] PATH_INFO: `/v3/projects/6706473c227a48888ef1e459916def43` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.893130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6706473c227a48888ef1e459916def43)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.895112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.915189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220baf-bc0b-4462-8585-0678b2c2fb77 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:11.919924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:11.920945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:11.921223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:11.921523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] PATH_INFO: `/v3/projects/6e2645870e154c17b0cf853a63a3eb06` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:11.922705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2645870e154c17b0cf853a63a3eb06)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:11.924640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:11.941479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6706473c227a48888ef1e459916def43', 'request_id': 'req-307fa2d5-9253-418e-a58a-3b65cd74cca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.948077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6706473c227a48888ef1e459916def43', 'request_id': 'req-307fa2d5-9253-418e-a58a-3b65cd74cca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.955983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6706473c227a48888ef1e459916def43', 'request_id': 'req-307fa2d5-9253-418e-a58a-3b65cd74cca9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.966794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] The token cache is being invalidate because project 6706473c227a48888ef1e459916def43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:11.967104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.967827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.969425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1041/2088] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:11 2022] DELETE /identity/v3/projects/6706473c227a48888ef1e459916def43 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:11.973977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2645870e154c17b0cf853a63a3eb06', 'request_id': 'req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.980496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2645870e154c17b0cf853a63a3eb06', 'request_id': 'req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:11.987880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2645870e154c17b0cf853a63a3eb06', 'request_id': 'req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.002570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] The token cache is being invalidate because project 6e2645870e154c17b0cf853a63a3eb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:12.002869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.003591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.006107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1048/2089] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:11 2022] DELETE /identity/v3/projects/6e2645870e154c17b0cf853a63a3eb06 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:12.010198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:12.011680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:12.081077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa2d5-9253-418e-a58a-3b65cd74cca9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:12.132286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:12.145509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:12.146396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:12.146677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:12.146983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] PATH_INFO: `/v3/users/a3c0752fca3f49afab256dc01a6f55a6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:12.148286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c0752fca3f49afab256dc01a6f55a6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:12.150277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:12.214395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c0752fca3f49afab256dc01a6f55a6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.215048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c0752fca3f49afab256dc01a6f55a6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.223019 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c0752fca3f49afab256dc01a6f55a6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.224837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c0752fca3f49afab256dc01a6f55a6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.241877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d5f9628-5f85-4362-ba19-f315939492ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c0752fca3f49afab256dc01a6f55a6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:12.256253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1042/2090] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:12 2022] DELETE /identity/v3/users/a3c0752fca3f49afab256dc01a6f55a6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:13.017637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.019068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:13.033452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3deef-8a96-4abc-82b6-f9a9f53ec62e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.039926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:13.131681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.132767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.133218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.133697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] PATH_INFO: `/v3/projects/e8623e27a43142cd82410b8b3257f64d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.135130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8623e27a43142cd82410b8b3257f64d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.138646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.285080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8623e27a43142cd82410b8b3257f64d', 'request_id': 'req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:13.311170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8623e27a43142cd82410b8b3257f64d', 'request_id': 'req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:13.338235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8623e27a43142cd82410b8b3257f64d', 'request_id': 'req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:13.368827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] The token cache is being invalidate because project e8623e27a43142cd82410b8b3257f64d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:13.369321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:13.370340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c5cd450-7afc-4542-a686-ebfc0fdd1abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:13.378523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1049/2091] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:13 2022] DELETE /identity/v3/projects/e8623e27a43142cd82410b8b3257f64d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:13.379418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.380827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.381265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.455497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:13.457486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:13.563055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1043/2092] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:13.571881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.572177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.572511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.605407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:13.606822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:13.616511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1050/2093] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:13.620269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.634925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140ef86-9a0d-44c2-944b-4c217b2cfecf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.642421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.643686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.643686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.643686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.644360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:13.644713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.647853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.664414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1044/2094] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:13.668692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.681206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b6474-5035-4cd5-8406-2b88ae705927 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.689372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.690069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.690353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.690619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.691365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.694013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.739477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1051/2095] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:13 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:13.743015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.755038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4984fa1-3c11-4570-b46b-591c3ee60966 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.850973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:13.862907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.868346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.868656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.869325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.870265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.873316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.903404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1045/2096] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:13 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:13.907593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.919191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea071b-cbfe-4595-98fe-aa6d6ae7a1a1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.924088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.924834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.925186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.925532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.926886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.928600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.948730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1052/2097] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:13.952673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:13.964307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fd882-e62a-498f-b707-ec8b1e5d4f5c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:13.970305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:13.971080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:13.971487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:13.971856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:13.972676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:13.974514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7577e0f-7e51-4167-9f3f-175dae6449eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:13.985426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1046/2098] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:13.990765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.000685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1438ae-d421-41dd-90b9-3e8a593b1e13 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.006036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.007077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.007220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.007579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] PATH_INFO: `/v3/projects/a6c0353d8fbe4861ad44ade90e963df3/users/6f4d6d641593485a9bda65a311f4cdf2/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.024732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6c0353d8fbe4861ad44ade90e963df3, user_id=6f4d6d641593485a9bda65a311f4cdf2, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.027482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.041340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1053/2099] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:13 2022] PUT /identity/v3/projects/a6c0353d8fbe4861ad44ade90e963df3/users/6f4d6d641593485a9bda65a311f4cdf2/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:14.052982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.053270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.053551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.080333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] MFA Rules not processed for user `6f4d6d641593485a9bda65a311f4cdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:14.081674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:14.151578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1047/2100] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:14.156408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.197820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:14.220889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac64f9-3b3c-48c4-98df-25251236a832 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.226587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.227195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.227594 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.227777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.228467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.230394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.263259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1054/2101] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:14 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:14.264742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.281127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab708f5-5689-4cff-80c0-fc6f364d93b1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.336387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.338055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:14.340826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.341493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.341774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.342049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.342782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.344734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.367429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1048/2102] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:14 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:14.374461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.388708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6081fab-8fea-4e4d-97d2-7f8a00673f89 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.394502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.396057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.396481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.396790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.397626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.399253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.412434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca0bd8-658a-44b6-b935-7f448008f678 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.412577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1049/2103] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:14.418538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.427842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.428742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.429185 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.430051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.430190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b449a5-2b0a-46e8-96c2-91f4071784e0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.432330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:14.435824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.436703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.437119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.437296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] PATH_INFO: `/v3/projects/6f6b4d832380414c9dc0c482a80607c9/users/585e7a907f5542b79cd6d37807806021/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.453938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6b4d832380414c9dc0c482a80607c9, user_id=585e7a907f5542b79cd6d37807806021, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.456672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.476300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1050/2104] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:14 2022] PUT /identity/v3/projects/6f6b4d832380414c9dc0c482a80607c9/users/585e7a907f5542b79cd6d37807806021/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:14.486134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.507074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.508582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.514163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:14.521227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1055/2105] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:14.575169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cb70b-c3b9-4a96-a037-170cd0e5b8c5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.579895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.580592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.580875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.581155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.581891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.584122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c45e21-cd79-424d-b8ba-0f7a62253dcf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.594078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1051/2106] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:14.598681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.609060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916da29-ead6-4484-81b9-6d58507800f3 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.613831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.614921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.615209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.615519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] PATH_INFO: `/v3/projects/6f6b4d832380414c9dc0c482a80607c9/users/585e7a907f5542b79cd6d37807806021/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.617509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6b4d832380414c9dc0c482a80607c9, user_id=585e7a907f5542b79cd6d37807806021, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.620218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.631174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1056/2107] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:14 2022] PUT /identity/v3/projects/6f6b4d832380414c9dc0c482a80607c9/users/585e7a907f5542b79cd6d37807806021/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:14.643395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.644120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.644590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.674674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] MFA Rules not processed for user `585e7a907f5542b79cd6d37807806021`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:14.676015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:14.756600 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1052/2108] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:14.768807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:14.780723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:14.870038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f589b2-8236-45f2-b3e5-bbe4f3722ff6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:14.873901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:14.874468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:14.875133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:14.875505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:14.890720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:14.890720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5a4e2-d06a-4bf3-99af-2db5ae660935 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:14.892992 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:14.899692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1057/2109] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:15.075799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:15.184563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d36459-22cd-4cac-bcb7-00b48ea7549e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:15.194103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:15.194979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:15.195298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:15.195749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] PATH_INFO: `/v3/users/6f4d6d641593485a9bda65a311f4cdf2` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:15.197445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4d6d641593485a9bda65a311f4cdf2)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:15.200132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:15.260552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:15.264090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4d6d641593485a9bda65a311f4cdf2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.265298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4d6d641593485a9bda65a311f4cdf2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.275332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4d6d641593485a9bda65a311f4cdf2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.276061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4d6d641593485a9bda65a311f4cdf2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.289545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4d6d641593485a9bda65a311f4cdf2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.303875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1053/2110] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:15 2022] DELETE /identity/v3/users/6f4d6d641593485a9bda65a311f4cdf2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:15.308630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:15.309165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:15.309281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:15.327404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:15.328608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:15.397124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1058/2111] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:15.406889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:15.408349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:15.469947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f747947-2375-4e71-bbc1-a37f09f4b322 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:15.474370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:15.475021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:15.475338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:15.475618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:15.485208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:15.486698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:15.497004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1054/2112] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:15.777424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:15.778865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:15.791803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8ea40-1471-46f5-921c-c5dc6f8127b0 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:15.796423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:15.797397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:15.797676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:15.797951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] PATH_INFO: `/v3/projects/a6c0353d8fbe4861ad44ade90e963df3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:15.799187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6c0353d8fbe4861ad44ade90e963df3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:15.801935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:15.860630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6c0353d8fbe4861ad44ade90e963df3', 'request_id': 'req-145efbf9-22b2-48a3-a132-809727144c51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.867135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6c0353d8fbe4861ad44ade90e963df3', 'request_id': 'req-145efbf9-22b2-48a3-a132-809727144c51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.873415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6c0353d8fbe4861ad44ade90e963df3', 'request_id': 'req-145efbf9-22b2-48a3-a132-809727144c51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.886762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] The token cache is being invalidate because project a6c0353d8fbe4861ad44ade90e963df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:15.887057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.887705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:15.889552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1059/2113] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:15 2022] DELETE /identity/v3/projects/a6c0353d8fbe4861ad44ade90e963df3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:15.894045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:15.905031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe0fe-c4c4-438f-8607-b8e0fea9ae92 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:15.979988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:15.980894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:15.981196 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:15.981594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] PATH_INFO: `/v3/users/585e7a907f5542b79cd6d37807806021` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:15.982785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] RBAC: Authorizing `identity:delete_user(user_id=585e7a907f5542b79cd6d37807806021)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:15.984785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:16.037199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585e7a907f5542b79cd6d37807806021'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.039170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585e7a907f5542b79cd6d37807806021'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.047474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585e7a907f5542b79cd6d37807806021'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.048253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585e7a907f5542b79cd6d37807806021'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.055839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:16.062850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3a80d9e5-549b-425c-a23c-7f4c38d7492a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585e7a907f5542b79cd6d37807806021'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.079272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1055/2114] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:15 2022] DELETE /identity/v3/users/585e7a907f5542b79cd6d37807806021 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:16.469443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:16.472147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:16.522172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145efbf9-22b2-48a3-a132-809727144c51 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:16.606860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:16.608020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:16.608467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:16.608856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] PATH_INFO: `/v3/projects/6f6b4d832380414c9dc0c482a80607c9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:16.610073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6b4d832380414c9dc0c482a80607c9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:16.612095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:16.657128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:16.688452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6b4d832380414c9dc0c482a80607c9', 'request_id': 'req-af4427db-e7f0-469b-ae11-26675467a758'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.698733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6b4d832380414c9dc0c482a80607c9', 'request_id': 'req-af4427db-e7f0-469b-ae11-26675467a758'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.719180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6b4d832380414c9dc0c482a80607c9', 'request_id': 'req-af4427db-e7f0-469b-ae11-26675467a758'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.737054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] The token cache is being invalidate because project 6f6b4d832380414c9dc0c482a80607c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:16.737359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.738060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af4427db-e7f0-469b-ae11-26675467a758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:16.739986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1060/2115] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:16 2022] DELETE /identity/v3/projects/6f6b4d832380414c9dc0c482a80607c9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:16.745099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:16.745571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:16.745711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:16.765120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:16.766593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:16.830660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1056/2116] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:16.839276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:16.839583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:16.839859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:16.859872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:16.860982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:16.868902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1061/2117] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:16.873239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:16.882923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed981d-cb25-4fdc-8036-d626d7edcba3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:16.888040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:16.888688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:16.888981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:16.889253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:16.890114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:16.890472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:16.892543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:16.905577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1057/2118] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:16.909873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:16.919720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71781f82-c9ed-439a-9d68-22b9cd4a40e9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:16.924338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:16.924984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:16.925265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:16.925540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:16.926490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:16.928506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efddf7d-2a65-4fae-b913-1a35c7e6d6d4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:16.960494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1062/2119] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:16 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:16.961865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:16.973893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1f5c6-9cbc-44e5-85b2-96b4358e25b6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.040472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.041182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.041545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.041829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.042519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.044497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b536a8b-60af-44b4-9041-01e5fd015393 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.056835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c755a0-cd77-4955-b2cd-37e4d6a0a257 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.057165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c755a0-cd77-4955-b2cd-37e4d6a0a257 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.057498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c755a0-cd77-4955-b2cd-37e4d6a0a257 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.058897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1063/2120] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:20:17.067326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1058/2121] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:16 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:17.070353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c755a0-cd77-4955-b2cd-37e4d6a0a257 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.084407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c755a0-cd77-4955-b2cd-37e4d6a0a257 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.089038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.089932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.090280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.091000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.091530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.093101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.093220 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.093621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.094002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.105722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1064/2122] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:17.109757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.115355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.116734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.121620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db835-35f4-4196-b259-97e767666b63 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.130890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.131805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.132372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.132751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.133612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.135411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.148280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1065/2123] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:17.149293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.151210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.163944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d549bb9-9fc2-4153-be4d-f4eb14ad20a7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.168742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.169875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.170228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.170570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] PATH_INFO: `/v3/users/eb6ed6f6c22945e8944ec2414bc2de87` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.171900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6ed6f6c22945e8944ec2414bc2de87)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.174145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.183203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:17.204494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1059/2124] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:17.205589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.217787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c1578-eee9-4f73-8b74-4aaef05be4fe None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.222985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.223978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.224382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.224724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] PATH_INFO: `/v3/projects/ea8dd93dfb5b47bb8309079d7325ece1/users/3e6307db2fa64b0cb98d568e7306581c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.235528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6ed6f6c22945e8944ec2414bc2de87'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.244083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6ed6f6c22945e8944ec2414bc2de87'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.244775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6ed6f6c22945e8944ec2414bc2de87'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.244888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8dd93dfb5b47bb8309079d7325ece1, user_id=3e6307db2fa64b0cb98d568e7306581c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.248083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee22797-4137-4272-90f0-3b19d2eeaefb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.253914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6ed6f6c22945e8944ec2414bc2de87'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.255253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2e4996f-10b3-478d-a2cf-e4c889c4ec4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6ed6f6c22945e8944ec2414bc2de87'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.266394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1060/2125] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:17 2022] PUT /identity/v3/projects/ea8dd93dfb5b47bb8309079d7325ece1/users/3e6307db2fa64b0cb98d568e7306581c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:17.267829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc55c4-74c7-4063-9dbd-6c1e127616f1 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.268116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc55c4-74c7-4063-9dbd-6c1e127616f1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.268616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc55c4-74c7-4063-9dbd-6c1e127616f1 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.269817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1061/2126] 10.222.0.39 () {54 vars in 960 bytes} [Thu May 26 03:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:20:17.276097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.276792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.277243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.277355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1066/2127] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:17 2022] DELETE /identity/v3/users/eb6ed6f6c22945e8944ec2414bc2de87 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:17.281028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.281508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.281918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.309655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.311006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.320655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] MFA Rules not processed for user `3e6307db2fa64b0cb98d568e7306581c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.321920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.367724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:17.406431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1062/2128] 10.222.0.39 () {58 vars in 1057 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:17.414367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.424365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d034cb-57fb-4792-94e4-724cdcb82a42 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.429432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1067/2129] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:17.429857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.430588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.430862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.431133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.435263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.440803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.442660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.447557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.449172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.462952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1063/2130] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:17.482063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.483577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.512218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584e4f8-7047-403a-bb7e-6e6990323274 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.516832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.517573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.518031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.518382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.528871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.530421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.541222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1068/2131] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:17.562142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.562318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc3b93-91b3-483c-a51f-d8d6e2b4c8c2 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.563911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.568010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.568937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.569221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.569497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] PATH_INFO: `/v3/projects/f3d32df9c64c414d9c93be35820927c9` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.570673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d32df9c64c414d9c93be35820927c9)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.572651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.578890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb76870-4521-4793-a079-35304b286eea service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.584040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.584997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.585275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.585558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] PATH_INFO: `/v3/users/3e6307db2fa64b0cb98d568e7306581c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.586924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6307db2fa64b0cb98d568e7306581c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.589087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.625792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d32df9c64c414d9c93be35820927c9', 'request_id': 'req-ce66d109-4a78-44fc-8667-e44296dba004'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.634717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d32df9c64c414d9c93be35820927c9', 'request_id': 'req-ce66d109-4a78-44fc-8667-e44296dba004'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.638369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:17.641766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6307db2fa64b0cb98d568e7306581c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.644681 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d32df9c64c414d9c93be35820927c9', 'request_id': 'req-ce66d109-4a78-44fc-8667-e44296dba004'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.650444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6307db2fa64b0cb98d568e7306581c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.651319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6307db2fa64b0cb98d568e7306581c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.658817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] The token cache is being invalidate because project f3d32df9c64c414d9c93be35820927c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:17.659111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.659863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-ce66d109-4a78-44fc-8667-e44296dba004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.659978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6307db2fa64b0cb98d568e7306581c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.660914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d61e8bd8-04f5-435a-be31-c21d439edc66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6307db2fa64b0cb98d568e7306581c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:17.661631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1064/2132] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:17 2022] DELETE /identity/v3/projects/f3d32df9c64c414d9c93be35820927c9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:17.666674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffdaef-ebe4-493c-9748-d04e5dc6802e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.666970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffdaef-ebe4-493c-9748-d04e5dc6802e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.667268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffdaef-ebe4-493c-9748-d04e5dc6802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.682121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1069/2133] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:17 2022] DELETE /identity/v3/users/3e6307db2fa64b0cb98d568e7306581c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:17.687843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f3ffdaef-ebe4-493c-9748-d04e5dc6802e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.688212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.688504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.688784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.689097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f3ffdaef-ebe4-493c-9748-d04e5dc6802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.708500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.709775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.748867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:17.757442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1065/2134] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:17.767400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.767764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.768043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.785762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1070/2135] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:17.786964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:17.788227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.796903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.798298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:17.800593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1066/2136] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:17.805861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.815645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419536-a407-43f1-a023-e5bddf224de3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.824945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.825618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.825886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.826190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.826924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:17.827254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.828901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.841867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1067/2137] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:17.846289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.857040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef6a0-0b01-4579-8fc2-0d7e2337655a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.862173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.862857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.863141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.864219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.865301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.866885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.888967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567a62b-efaa-47cb-be21-5b672c28db6f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.893199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.893848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.894138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.894409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.896579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1068/2138] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:17.901362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:17.909043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.911335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:17.922022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1071/2139] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:17.974771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df654e7a-5c48-49a9-a290-f1f45ff4abbb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:17.979892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:17.980568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:17.980886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:17.981106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:17.981788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:17.983793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.016495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1069/2140] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:17 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:18.019926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.031026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7830787-100e-4d61-8234-971db38ebb8e service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.037978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.038728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.039156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.039523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.040381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.042278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.079982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1072/2141] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:18.085167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.098799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcdf61-a4ab-46c6-8813-8d9e2acc3a59 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.122875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.124309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.124842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.125364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.127086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.127988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8043e2-ed5e-45f9-a778-e5431eedb6d7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.140470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1070/2142] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:18.145602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.157111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba45272-1a2a-4c38-952b-e00bf0408ef7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.163646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.164744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.165192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.165604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] PATH_INFO: `/v3/projects/2d9bfd3a2b544d74bb581b5bd13e8ecc/users/7708b4816bcf4301ba7cc826d533aefc/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.191588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9bfd3a2b544d74bb581b5bd13e8ecc, user_id=7708b4816bcf4301ba7cc826d533aefc, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.195823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.217381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1073/2143] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:18 2022] PUT /identity/v3/projects/2d9bfd3a2b544d74bb581b5bd13e8ecc/users/7708b4816bcf4301ba7cc826d533aefc/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:18.222697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.223032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.223353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.242066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:18.243438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.260977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.262770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.263877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:18.325639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1071/2144] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:18.337628 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2517-8939-4b8b-a656-39346c4c9df9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.340393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.341959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.342494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.343379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.343661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.343932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] PATH_INFO: `/v3/projects/ea8dd93dfb5b47bb8309079d7325ece1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.345099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8dd93dfb5b47bb8309079d7325ece1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.347112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.414189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8dd93dfb5b47bb8309079d7325ece1', 'request_id': 'req-f25b35cd-ca28-4973-8476-6b1aff265e06'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:18.416143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:18.418877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5a2e4-3e7c-44bf-a304-860e05c9605f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.426101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8dd93dfb5b47bb8309079d7325ece1', 'request_id': 'req-f25b35cd-ca28-4973-8476-6b1aff265e06'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:18.436446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8dd93dfb5b47bb8309079d7325ece1', 'request_id': 'req-f25b35cd-ca28-4973-8476-6b1aff265e06'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:18.451705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] The token cache is being invalidate because project ea8dd93dfb5b47bb8309079d7325ece1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:18.452066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:18.452783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f25b35cd-ca28-4973-8476-6b1aff265e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:18.454710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1074/2145] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:18 2022] DELETE /identity/v3/projects/ea8dd93dfb5b47bb8309079d7325ece1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:18.459964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9a1b-b3ba-4dda-b456-706ec94cca17 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.460250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9a1b-b3ba-4dda-b456-706ec94cca17 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.460598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9a1b-b3ba-4dda-b456-706ec94cca17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.478343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.478904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.479224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.479507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.481362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9e1f9a1b-b3ba-4dda-b456-706ec94cca17 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:18.481486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.482763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9e1f9a1b-b3ba-4dda-b456-706ec94cca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.548485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1075/2146] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:18.551768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.553235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.557271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.557567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.557857 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.563615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1072/2147] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:18.576950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:18.578148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:18.587209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1076/2148] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:18.591591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.601795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dd8d0-3347-4152-a553-5932ae386927 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.606539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.607184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.607490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.607760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.608470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:18.608771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.610360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.625885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1073/2149] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:18.627008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.637834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d7d47-6d41-4b59-b9da-fbb5842c7f21 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.644205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.644963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.645245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.645536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.646225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.648269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.677602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1077/2150] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:18.681943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.693856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2d4d5-e226-4b59-9d13-18f86a036525 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.707607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:18.762446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.763126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.763431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.763721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.764938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.767155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.788430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1074/2151] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:18.792850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.804173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4fc1d-c645-40bc-83d7-edf9a56f1b9d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.809790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.810445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.810932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.811098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.811935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.813544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.825412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1078/2152] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:18.829408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.840553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50423041-05c3-4cad-816f-715203dab5d4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.845974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.846641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.846937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.847287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.848675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.850438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478b658-5400-4d52-8aa3-d0a60386bf45 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.861120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1075/2153] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:18.865602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:18.877080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4aaeb0-0674-4416-b858-bce47c70ef36 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:18.884008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:18.884905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.885192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.885472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] PATH_INFO: `/v3/projects/1eab77e85b2f4ed190ae0dd81363feab/users/61540458ac4941b3b6ae292b4a873f57/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.903921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eab77e85b2f4ed190ae0dd81363feab, user_id=61540458ac4941b3b6ae292b4a873f57, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:18.907357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:18.923872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1079/2154] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:18 2022] PUT /identity/v3/projects/1eab77e85b2f4ed190ae0dd81363feab/users/61540458ac4941b3b6ae292b4a873f57/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:18.934695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:18.934908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:18.935179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:18.964042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] MFA Rules not processed for user `61540458ac4941b3b6ae292b4a873f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:18.965158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:19.036734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1076/2155] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:19.041232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.116007 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7d857-a5de-4fc8-a19c-d9a5279806d1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.120863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.121569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.121855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.122130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.122816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.124761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.146034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:19.153473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1080/2156] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:19.157695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.172920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada5aa8-d479-494a-b14d-938c72a84745 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.248123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.249000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.249349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.249722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.250526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.252904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.288521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1077/2157] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:19.290614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.328010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cca0d-86a5-4297-977f-2a0e3e42d238 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.336210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.337402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.337894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.338551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.339654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.342176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.367396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.383258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1081/2158] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:19.385359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8602e8-845a-4eb1-9ea4-60e4ec917715 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.391587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.393043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.393890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.395832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.396886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.399265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f99dd-ac99-42c2-8160-3006363a7517 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.414104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1078/2159] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:19.418917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.434926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b017590-d689-4817-bd39-f54bb8ed17ad admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.443081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.444678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.445127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.445772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] PATH_INFO: `/v3/projects/553bf3f6770c497091ab6648b2024ce4/users/e6aef9075c824ddc8617a6cd805a25b0/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.465189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=553bf3f6770c497091ab6648b2024ce4, user_id=e6aef9075c824ddc8617a6cd805a25b0, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.468231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.486960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1082/2160] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:19 2022] PUT /identity/v3/projects/553bf3f6770c497091ab6648b2024ce4/users/e6aef9075c824ddc8617a6cd805a25b0/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:19.500129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.500530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.500872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.514716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:19.540398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.542352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:19.561607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] MFA Rules not processed for user `e6aef9075c824ddc8617a6cd805a25b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:19.563535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:19.654613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1079/2161] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:19.658933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.683211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa24a-f239-4338-b2ad-fc89d3c45f5c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.689720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.691147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.691611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.692398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] PATH_INFO: `/v3/users/18787fd1d17e4ae69e911d895342f746` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.694716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18787fd1d17e4ae69e911d895342f746)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.697770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.746089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6bcc3-3b49-4ed9-89c6-505110ad700d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.755484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:19.756292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.756636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.757424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.758112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:19.760226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:19.768025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18787fd1d17e4ae69e911d895342f746'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:19.777299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18787fd1d17e4ae69e911d895342f746'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:19.778431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18787fd1d17e4ae69e911d895342f746'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:19.790061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18787fd1d17e4ae69e911d895342f746'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:19.791054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a7f45ff6-8abe-417c-b99a-d168a00f1f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18787fd1d17e4ae69e911d895342f746'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:19.794343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1080/2162] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:19.798675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.803519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:19.820862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1083/2163] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:19 2022] DELETE /identity/v3/users/18787fd1d17e4ae69e911d895342f746 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:19.826346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:19.826517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:19.827037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:19.848061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:19.849284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:19.904284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:19.913249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672049d-343f-4f08-ade3-608bfff4266c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:19.942441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:19.942751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1084/2164] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:19.952879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:19.954787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.005899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.006614 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.007548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.007819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.008759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.010941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.041653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11802982-235f-49a6-b7f3-0ccb5cc2935c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.045790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.046444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.047179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.047342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1081/2165] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:19 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:20.047427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.051726 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.059783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.061235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.072765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f809c4-ff5c-44a2-9fcd-1c33b95c5cf8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.079358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1085/2166] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:20.087029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.087807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.088077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.088351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.089093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.091146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.110308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1082/2167] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:20.114453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.137253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138ed95-b568-471d-a2c8-4672c0aa5f90 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.145020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.145782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.146247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.146869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] PATH_INFO: `/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c/users/f516632d25e443c68070f57bd3ca73a0/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.174356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8f4fb1417e49e198f81c22fd9fbe5c, user_id=f516632d25e443c68070f57bd3ca73a0, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.177370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.208171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.208906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1086/2168] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:20 2022] PUT /identity/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c/users/f516632d25e443c68070f57bd3ca73a0/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:20.223892 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a5bca-d78e-4fe2-9c71-c2923f9106a8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.322976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.323954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.324417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.324837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.325770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.327793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de0a3b-9127-4098-9c79-c46f337f0b72 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.338834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1083/2169] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:20.343323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.355782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6af9-7c1a-4539-9efd-96462a1259e8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.360645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.361701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.362044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.362385 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] PATH_INFO: `/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c/users/f516632d25e443c68070f57bd3ca73a0/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.367348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8f4fb1417e49e198f81c22fd9fbe5c, user_id=f516632d25e443c68070f57bd3ca73a0, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.367348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.392318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1087/2170] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:20 2022] PUT /identity/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c/users/f516632d25e443c68070f57bd3ca73a0/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:20.395467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.395803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.396078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.429135 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] MFA Rules not processed for user `f516632d25e443c68070f57bd3ca73a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:20.431031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.502910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.509446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.577745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1084/2171] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:20.595283 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.596837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.597371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85348e2b-955b-4f7e-843d-28464c40a48c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.602430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.603340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.603622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.603894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] PATH_INFO: `/v3/projects/c10270d360324ed2974748fbae02affd` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.605112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c10270d360324ed2974748fbae02affd)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.607045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.637789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:20.689446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c10270d360324ed2974748fbae02affd', 'request_id': 'req-d6dddbda-7904-427f-a97d-95c1e864a973'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.692568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:20.697839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c10270d360324ed2974748fbae02affd', 'request_id': 'req-d6dddbda-7904-427f-a97d-95c1e864a973'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.698350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5ae50-d0c3-41ad-aef7-c4f7499aa8c8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.706725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c10270d360324ed2974748fbae02affd', 'request_id': 'req-d6dddbda-7904-427f-a97d-95c1e864a973'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.725078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] The token cache is being invalidate because project c10270d360324ed2974748fbae02affd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:20.725078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.725987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.728544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1088/2172] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:20 2022] DELETE /identity/v3/projects/c10270d360324ed2974748fbae02affd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:20.733794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:20.735661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.769302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.770707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.770990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.771276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.773285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:20.849237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dddbda-7904-427f-a97d-95c1e864a973 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:20.850650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.852293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.860267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:20.863579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1085/2173] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:20.866617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:20.867022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:20.867435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] PATH_INFO: `/v3/users/9965cc3fc6a8428190553b638e327885` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:20.869094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9965cc3fc6a8428190553b638e327885)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:20.873188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:20.955161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9965cc3fc6a8428190553b638e327885'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.964285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9965cc3fc6a8428190553b638e327885'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.965035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9965cc3fc6a8428190553b638e327885'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.978795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9965cc3fc6a8428190553b638e327885'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:20.979701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-178435ae-2c7a-4a7b-ba46-0d864d2acf8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9965cc3fc6a8428190553b638e327885'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.007430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1089/2174] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:20 2022] DELETE /identity/v3/users/9965cc3fc6a8428190553b638e327885 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:21.445469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:21.463451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78cc1-373b-49ab-b75e-effbfdf43cda service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:21.578920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:21.579816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:21.580152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:21.580430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] PATH_INFO: `/v3/projects/8eb1a38386fa4e21bae952472a3ee470` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:21.581836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb1a38386fa4e21bae952472a3ee470)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:21.584450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:21.636372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:21.648116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb1a38386fa4e21bae952472a3ee470', 'request_id': 'req-608e52c9-2dc7-42a2-8f13-f473971bad11'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.655760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb1a38386fa4e21bae952472a3ee470', 'request_id': 'req-608e52c9-2dc7-42a2-8f13-f473971bad11'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.670372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb1a38386fa4e21bae952472a3ee470', 'request_id': 'req-608e52c9-2dc7-42a2-8f13-f473971bad11'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.686792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] The token cache is being invalidate because project 8eb1a38386fa4e21bae952472a3ee470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:21.687084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.687835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-608e52c9-2dc7-42a2-8f13-f473971bad11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:21.694352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1086/2175] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:21 2022] DELETE /identity/v3/projects/8eb1a38386fa4e21bae952472a3ee470 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:21.697407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:21.697854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:21.698042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:21.720434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:21.721655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:21.792618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:21.812598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1090/2176] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:21.821166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:21.821655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:21.821932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:21.842424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:21.844003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:21.856767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1087/2177] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:21.861074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:21.872149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcff288-8a01-4e05-8788-a65cb12b85c3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:21.877693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:21.878512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:21.879562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:21.880135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:21.881131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:21.881443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:21.883092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:21.912717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1091/2178] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:21.913178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:21.923295 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71408a4b-4caa-430b-b0a6-9330552f8a77 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:21.929982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:21.930665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:21.931094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:21.931869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:21.932420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:21.934497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:21.975287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1088/2179] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:21 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:21.977909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:21.990336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578866bc-888e-4422-8887-fdbf76805220 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:22.058227 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:22.058912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.059191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.059507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.060195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:22.062826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:22.090172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1092/2180] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:21 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:22.093916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:22.112120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7661ce6-9edb-4ed3-a5dc-b013588670fa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:22.121557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:22.122314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.122582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.122861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.123623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:22.125528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:22.145517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1089/2181] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:22.149701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:22.163408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523daf6-597e-40c0-832b-b0e6d22286b3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:22.170056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:22.170755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.171458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.171458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.172112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:22.173736 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b226f-96b1-428f-bb5e-4568f3ab6867 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:22.189291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1093/2182] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:22.198259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:22.212267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c6dc-22a7-42d7-a05e-d7e94e466bb0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:22.222015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:22.222015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.222015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.222015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] PATH_INFO: `/v3/projects/5b8a415a2a93492fa4f6b3c9acc0b080/users/50d1dfc235cc4c5e8f375b4dbc1d0b51/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.238950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8a415a2a93492fa4f6b3c9acc0b080, user_id=50d1dfc235cc4c5e8f375b4dbc1d0b51, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:22.243867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:22.264569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1090/2183] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:22 2022] PUT /identity/v3/projects/5b8a415a2a93492fa4f6b3c9acc0b080/users/50d1dfc235cc4c5e8f375b4dbc1d0b51/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:22.272255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1691fe-c831-46dc-97b9-c2ca3ecbb351 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.272798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1691fe-c831-46dc-97b9-c2ca3ecbb351 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.273347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1691fe-c831-46dc-97b9-c2ca3ecbb351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.300084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cb1691fe-c831-46dc-97b9-c2ca3ecbb351 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:22.302880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb1691fe-c831-46dc-97b9-c2ca3ecbb351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:22.416202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1094/2184] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:22.438244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:22.440328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:22.556557 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981083f-465e-46a2-b8c3-ce4363cb4299 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:22.563028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:22.564204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:22.565491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:22.566017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:22.583489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:22.595775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c718b1f-8315-4515-93a4-05e202c2ce50 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:22.609603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1091/2185] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:23.489144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.489659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.489956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.529736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d775-c1de-43ea-973e-4ee137c0c586 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.530197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d775-c1de-43ea-973e-4ee137c0c586 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.530291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8d775-c1de-43ea-973e-4ee137c0c586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.542436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] MFA Rules not processed for user `50d1dfc235cc4c5e8f375b4dbc1d0b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:23.543716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.572229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ecf8d775-c1de-43ea-973e-4ee137c0c586 None None] MFA Rules not processed for user `7708b4816bcf4301ba7cc826d533aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:23.573473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ecf8d775-c1de-43ea-973e-4ee137c0c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.663152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:23.677805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:23.682988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1095/2186] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:23.696035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:23.697766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.719550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1092/2187] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:23.719550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.721486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.721603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.742540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] MFA Rules not processed for user `7708b4816bcf4301ba7cc826d533aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:23.743832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.757928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1093/2188] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:23.766723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:23.768192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.803584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980cd30-dc56-45d7-821d-793fe1a24b8c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:23.813639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:23.814466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.814783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.815037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.826298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:23.828475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:23.843931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1096/2189] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:23.845036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:23.846608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.860993 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ed604-52ec-4480-ad37-1e1afc7aace7 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:23.865198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:23.865938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.866301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.866688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.868648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:23.904160 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1469089-6b6f-4630-8f76-00489f040c20 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:23.914096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:23.915390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:23.915731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:23.915934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:23.927800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:23.929445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ed253-1f31-4978-b645-802acef5e6f2 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:23.939322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1094/2190] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:23.966092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:23.967812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:23.989999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1097/2191] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:24.949459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aebac-3889-47c6-a176-16673af9470f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:24.950597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aebac-3889-47c6-a176-16673af9470f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:24.951101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8aebac-3889-47c6-a176-16673af9470f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:24.985011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e8aebac-3889-47c6-a176-16673af9470f None None] MFA Rules not processed for user `7708b4816bcf4301ba7cc826d533aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:24.986964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e8aebac-3889-47c6-a176-16673af9470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:24.988164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:25.002131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1095/2192] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:25.009923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be633e53-96a2-4c75-a76c-f28794872112 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:25.021073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:25.021815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:25.022155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:25.022495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:25.036614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:25.214739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:25.216340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:25.239059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1098/2193] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:25.421523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a38c-8cfe-4272-a849-2262e61697a4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:25.421843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a38c-8cfe-4272-a849-2262e61697a4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:25.422156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352a38c-8cfe-4272-a849-2262e61697a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:25.447467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-9352a38c-8cfe-4272-a849-2262e61697a4 None None] MFA Rules not processed for user `50d1dfc235cc4c5e8f375b4dbc1d0b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:25.449052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9352a38c-8cfe-4272-a849-2262e61697a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:25.463070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1096/2194] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:27.279707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:27.281440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:27.305785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f31f4f-ba14-49d4-98d1-7279943bd939 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:27.312466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:27.313800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:27.314226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:27.315748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] PATH_INFO: `/v3/users/1039efdb9a874b008b2d25b85d3ba510` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:27.317780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1039efdb9a874b008b2d25b85d3ba510)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:27.321026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:27.393329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:27.453348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1039efdb9a874b008b2d25b85d3ba510'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:27.473095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1039efdb9a874b008b2d25b85d3ba510'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:27.474883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1039efdb9a874b008b2d25b85d3ba510'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:27.505008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1039efdb9a874b008b2d25b85d3ba510'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:27.509838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1039efdb9a874b008b2d25b85d3ba510'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:27.568333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1099/2195] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:27 2022] DELETE /identity/v3/users/1039efdb9a874b008b2d25b85d3ba510 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:27.572961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:27.573480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:27.573572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:27.596439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:27.598668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:27.688543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1097/2196] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:27.699377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:27.701165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:27.791420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d64a3-89f4-4179-8e18-fcdefa02bdfa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:27.796462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:27.797623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:27.798013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:27.798558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:27.816382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:27.819716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:27.834876 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1100/2197] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:29.048236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:29.052604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:29.092214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeecdd-a4a8-49b1-be38-5491c755916b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:29.098347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:29.099638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:29.100069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:29.100518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] PATH_INFO: `/v3/projects/74032b07d42e429c85df69f2c4a3d72b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:29.102261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=74032b07d42e429c85df69f2c4a3d72b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:29.104876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:29.303099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74032b07d42e429c85df69f2c4a3d72b', 'request_id': 'req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.324383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74032b07d42e429c85df69f2c4a3d72b', 'request_id': 'req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.326479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:29.337247 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74032b07d42e429c85df69f2c4a3d72b', 'request_id': 'req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.350213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] The token cache is being invalidate because project 74032b07d42e429c85df69f2c4a3d72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:29.350454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.351288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.356482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1098/2198] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:29 2022] DELETE /identity/v3/projects/74032b07d42e429c85df69f2c4a3d72b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:29.359600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:29.374885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494568cb-6d60-4c2e-b8e8-d65e2313fc49 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:29.546304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:29.547602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:29.548291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:29.548780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] PATH_INFO: `/v3/users/125f89e5e3b3400cab3a01e809f2618a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:29.550372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=125f89e5e3b3400cab3a01e809f2618a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:29.553063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:29.645387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125f89e5e3b3400cab3a01e809f2618a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.646814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:29.658558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125f89e5e3b3400cab3a01e809f2618a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.660801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125f89e5e3b3400cab3a01e809f2618a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.673192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125f89e5e3b3400cab3a01e809f2618a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.674381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fc1c4eac-47a5-4f9b-afe4-f884e8638e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125f89e5e3b3400cab3a01e809f2618a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:29.713388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1101/2199] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:29 2022] DELETE /identity/v3/users/125f89e5e3b3400cab3a01e809f2618a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:30.116872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.118449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:30.137086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f0e49-90b6-4879-b67e-cc67d277e4ae admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.196943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.197799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.198143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.198598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] PATH_INFO: `/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.200377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb9c2d0ac864ee9a51841ab5498d165)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.202927 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.253177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb9c2d0ac864ee9a51841ab5498d165', 'request_id': 'req-fd18d86e-f1e9-45bd-af1e-c919163f3aab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:30.260136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb9c2d0ac864ee9a51841ab5498d165', 'request_id': 'req-fd18d86e-f1e9-45bd-af1e-c919163f3aab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:30.267933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb9c2d0ac864ee9a51841ab5498d165', 'request_id': 'req-fd18d86e-f1e9-45bd-af1e-c919163f3aab'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:30.278618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] The token cache is being invalidate because project 9fb9c2d0ac864ee9a51841ab5498d165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:30.278903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:30.279542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-fd18d86e-f1e9-45bd-af1e-c919163f3aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:30.281974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1099/2200] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:30 2022] DELETE /identity/v3/projects/9fb9c2d0ac864ee9a51841ab5498d165 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:30.288134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.288423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.288700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.308244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:30.309546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:30.372762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1102/2201] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:30.381821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.382106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.382382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.402065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:30.404150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:30.414281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1100/2202] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:30.418465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.428837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f5aa6-2341-446b-866d-27b752483db7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.433508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.434155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.434433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.434701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.435452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:30.435757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.437378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.449828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1103/2203] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:30.453984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.463139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5c270-a0fe-4a0b-9b81-dd7dc1f42e70 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.467667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.468307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.468604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.468873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.469561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.471500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.499122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1101/2204] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:30.504068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.514332 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fa00d-ad0d-4eca-97cd-2e345f281c84 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.570393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.571056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.571373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.571649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.572324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.574238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.594100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1104/2205] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:30.598731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.609198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd8729-39bc-484c-a5f7-c50cd02ebe39 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.614564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.615213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.615528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.615806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.616526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.618128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.628292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1102/2206] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:30.632186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.642436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c7b54-6267-4e44-9be0-78b945e31d29 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.647464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.648111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.648389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.648661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.649376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.650954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279f0e4-b779-4054-9d7c-f8f7a77f81ac admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.661337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1105/2207] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:30.665319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.675140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb35e-07fe-4612-836f-3ccec40a33ef admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.679893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.680754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.681049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.681311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] PATH_INFO: `/v3/projects/0dd78eea617c452bb894be4580c3309c/users/0f11442b95f646418f4e6c527c6b0cf8/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.696745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd78eea617c452bb894be4580c3309c, user_id=0f11442b95f646418f4e6c527c6b0cf8, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.699462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.712575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1103/2208] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:30 2022] PUT /identity/v3/projects/0dd78eea617c452bb894be4580c3309c/users/0f11442b95f646418f4e6c527c6b0cf8/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:30.722697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.723001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.723304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.749504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] MFA Rules not processed for user `0f11442b95f646418f4e6c527c6b0cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:30.750608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:30.770778 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:30.819903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1106/2209] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:30.824391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.835200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:30.889049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541381-fe07-4b78-a013-5d7a183f7e45 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:30.895370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:30.896327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:30.896649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:30.897009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:30.897648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:30.899716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:30.931594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1104/2210] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:30.935299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:30.961174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb6968-1080-46bb-a0ef-4656d217ecc9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.054645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.056725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.057919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.058353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.059695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.064292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.101158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1107/2211] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:30 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:31.101981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:31.116171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12404f24-fba3-4350-a403-60a6ab78dee7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.123696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.125246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.125680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.126143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.127099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.129738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.149881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1105/2212] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:31.150401 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:31.162920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce890571-5a80-4392-823b-8c8ad558029d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.168584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.169657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.170012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.170364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] PATH_INFO: `/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a/users/90687fedf0bf4c5abaea15c9d9f77857/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.191671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d640952ca4a14d9ab0c1fe397296ee5a, user_id=90687fedf0bf4c5abaea15c9d9f77857, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.195086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.220601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1108/2213] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:31 2022] PUT /identity/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a/users/90687fedf0bf4c5abaea15c9d9f77857/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:31.222039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:31.237289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9633dfc-7d03-452b-acd0-d6bb3686cabb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.333693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.334603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.334899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.335291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.336277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.338072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083caf18-d581-4fb1-8b12-4b072c37d8a8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.361430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1106/2214] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:31.362472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:31.381906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28fe8b-d23b-4fdd-8540-af0a17e2cdce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.395344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.397226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.397669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.398155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] PATH_INFO: `/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a/users/90687fedf0bf4c5abaea15c9d9f77857/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.402940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d640952ca4a14d9ab0c1fe397296ee5a, user_id=90687fedf0bf4c5abaea15c9d9f77857, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.407730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.422001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1109/2215] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:31 2022] PUT /identity/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a/users/90687fedf0bf4c5abaea15c9d9f77857/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:31.433323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.433781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.433953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.466544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] MFA Rules not processed for user `90687fedf0bf4c5abaea15c9d9f77857`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:31.468076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:31.561212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1107/2216] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:31.575121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:31.576848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:31.654863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96c788-1ac2-4920-97d7-18189e699c90 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:31.659757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:31.661051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:31.661517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:31.661991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:31.673320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:31.675062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:31.686383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1110/2217] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:32.045799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:32.047367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:32.057613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:32.150017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab4942-fa02-480c-a5cf-abe6944ed1e2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:32.155704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:32.156710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.157204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.157721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] PATH_INFO: `/v3/users/61540458ac4941b3b6ae292b4a873f57` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.158875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=61540458ac4941b3b6ae292b4a873f57)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:32.162375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:32.231017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:32.245446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e3519-9926-4018-bce8-43945019b2e9 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:32.252328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:32.255545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.256510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.264308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] PATH_INFO: `/v3/users/0f11442b95f646418f4e6c527c6b0cf8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.266515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f11442b95f646418f4e6c527c6b0cf8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:32.271514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:32.279108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61540458ac4941b3b6ae292b4a873f57'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.280467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61540458ac4941b3b6ae292b4a873f57'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.294615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:32.302853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61540458ac4941b3b6ae292b4a873f57'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.304120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61540458ac4941b3b6ae292b4a873f57'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.330394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b6ddd29e-f222-44ea-b874-106bcd45a5ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61540458ac4941b3b6ae292b4a873f57'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.364253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f11442b95f646418f4e6c527c6b0cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.376195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f11442b95f646418f4e6c527c6b0cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.376376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1108/2218] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:32 2022] DELETE /identity/v3/users/61540458ac4941b3b6ae292b4a873f57 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:32.377356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f11442b95f646418f4e6c527c6b0cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.377978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.378279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.379446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.398205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f11442b95f646418f4e6c527c6b0cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.398839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c81994f0-5dc1-483e-934d-615164a19fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f11442b95f646418f4e6c527c6b0cf8'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:32.416808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:32.418427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:32.426035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1111/2219] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:32 2022] DELETE /identity/v3/users/0f11442b95f646418f4e6c527c6b0cf8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:32.431735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.432308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.432404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.466606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:32.468707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:32.551037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1109/2220] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:32.571335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:32.574234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:32.575579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1112/2221] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:32.594682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:32.597909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:32.636838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:32.724156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5606dd-354d-4d68-9931-5acbe6b0f991 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:32.730119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:32.742703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.742997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.743451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.756320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:32.758087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:32.770854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1110/2222] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:32.775436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab268d6-3c04-4748-b00b-139550e13ea4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:32.780121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:32.781089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:32.781561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:32.782043 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:32.794169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:32.796008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:32.808443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1113/2223] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:33.207752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:33.210795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:33.230992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05395e2-e43d-462d-b122-941c305da5a9 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:33.236127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:33.236989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:33.237472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:33.238432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] PATH_INFO: `/v3/projects/1eab77e85b2f4ed190ae0dd81363feab` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:33.242127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eab77e85b2f4ed190ae0dd81363feab)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:33.244327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:33.291094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:33.294717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:33.319490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eab77e85b2f4ed190ae0dd81363feab', 'request_id': 'req-26aca71f-91d3-4d66-80f4-83e4c2dd8761'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.326879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eab77e85b2f4ed190ae0dd81363feab', 'request_id': 'req-26aca71f-91d3-4d66-80f4-83e4c2dd8761'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.340083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2fa5f-fd33-455f-9dc5-3f2333bdbe98 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:33.341314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eab77e85b2f4ed190ae0dd81363feab', 'request_id': 'req-26aca71f-91d3-4d66-80f4-83e4c2dd8761'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.365471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] The token cache is being invalidate because project 1eab77e85b2f4ed190ae0dd81363feab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:33.367664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.371164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.374694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1111/2224] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:33 2022] DELETE /identity/v3/projects/1eab77e85b2f4ed190ae0dd81363feab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:33.379386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:33.381531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:33.390862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:33.429356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:33.430418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:33.430704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:33.430978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] PATH_INFO: `/v3/projects/0dd78eea617c452bb894be4580c3309c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:33.432187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd78eea617c452bb894be4580c3309c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:33.434115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:33.457177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca71f-91d3-4d66-80f4-83e4c2dd8761 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:33.461925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:33.462802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:33.463425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:33.463425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] PATH_INFO: `/v3/users/e6aef9075c824ddc8617a6cd805a25b0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:33.464575 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6aef9075c824ddc8617a6cd805a25b0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:33.466518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:33.499461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd78eea617c452bb894be4580c3309c', 'request_id': 'req-d11b5a4f-8546-4244-a46e-0ab9180ea456'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.506244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd78eea617c452bb894be4580c3309c', 'request_id': 'req-d11b5a4f-8546-4244-a46e-0ab9180ea456'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.514510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd78eea617c452bb894be4580c3309c', 'request_id': 'req-d11b5a4f-8546-4244-a46e-0ab9180ea456'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.524856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6aef9075c824ddc8617a6cd805a25b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.525755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6aef9075c824ddc8617a6cd805a25b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.530604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:33.533305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] The token cache is being invalidate because project 0dd78eea617c452bb894be4580c3309c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:33.533676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.535507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.536139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6aef9075c824ddc8617a6cd805a25b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.537823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6aef9075c824ddc8617a6cd805a25b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.537940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1114/2225] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:33 2022] DELETE /identity/v3/projects/0dd78eea617c452bb894be4580c3309c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:33.542059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:33.544375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:33.554837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6aef9075c824ddc8617a6cd805a25b0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.575066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1112/2226] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:33 2022] DELETE /identity/v3/users/e6aef9075c824ddc8617a6cd805a25b0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:33.650204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b5a4f-8546-4244-a46e-0ab9180ea456 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:33.736824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:33.737812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:33.738276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:33.738567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] PATH_INFO: `/v3/users/90687fedf0bf4c5abaea15c9d9f77857` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:33.739937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=90687fedf0bf4c5abaea15c9d9f77857)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:33.742158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:33.756414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:33.843462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90687fedf0bf4c5abaea15c9d9f77857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.854278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90687fedf0bf4c5abaea15c9d9f77857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.855603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90687fedf0bf4c5abaea15c9d9f77857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.867597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90687fedf0bf4c5abaea15c9d9f77857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.868611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90687fedf0bf4c5abaea15c9d9f77857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:33.914921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1115/2227] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:33 2022] DELETE /identity/v3/users/90687fedf0bf4c5abaea15c9d9f77857 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:34.370931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.372442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:34.386858 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc48a4b-1056-420c-b658-656dad4668d4 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:34.452727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:34.453673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.454111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.454576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] PATH_INFO: `/v3/projects/553bf3f6770c497091ab6648b2024ce4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.456217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] RBAC: Authorizing `identity:delete_project(project_id=553bf3f6770c497091ab6648b2024ce4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:34.459005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:34.527574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553bf3f6770c497091ab6648b2024ce4', 'request_id': 'req-2b329179-1076-4412-a2bd-075e812a655e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.535971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553bf3f6770c497091ab6648b2024ce4', 'request_id': 'req-2b329179-1076-4412-a2bd-075e812a655e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.546906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553bf3f6770c497091ab6648b2024ce4', 'request_id': 'req-2b329179-1076-4412-a2bd-075e812a655e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.556769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.558453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:34.560162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] The token cache is being invalidate because project 553bf3f6770c497091ab6648b2024ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:34.560780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.561757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.563914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1113/2228] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:34 2022] DELETE /identity/v3/projects/553bf3f6770c497091ab6648b2024ce4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:34.568106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.570145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:34.573564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc3f7e-701d-462c-98fc-646f58c164ce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:34.641751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:34.642735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.643090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.643463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] PATH_INFO: `/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.645384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d640952ca4a14d9ab0c1fe397296ee5a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:34.645715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329179-1076-4412-a2bd-075e812a655e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:34.647486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:34.653106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:34.654588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.655065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.655593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] PATH_INFO: `/v3/users/f516632d25e443c68070f57bd3ca73a0` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.657583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f516632d25e443c68070f57bd3ca73a0)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:34.660719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:34.662308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:34.697881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd640952ca4a14d9ab0c1fe397296ee5a', 'request_id': 'req-5fa96a96-8466-4d36-a5b1-23cacfa6078b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.703711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd640952ca4a14d9ab0c1fe397296ee5a', 'request_id': 'req-5fa96a96-8466-4d36-a5b1-23cacfa6078b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.710037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd640952ca4a14d9ab0c1fe397296ee5a', 'request_id': 'req-5fa96a96-8466-4d36-a5b1-23cacfa6078b'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.712316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f516632d25e443c68070f57bd3ca73a0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.712984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f516632d25e443c68070f57bd3ca73a0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.720131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f516632d25e443c68070f57bd3ca73a0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.720791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f516632d25e443c68070f57bd3ca73a0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.722874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] The token cache is being invalidate because project d640952ca4a14d9ab0c1fe397296ee5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:34.723165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.723851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5fa96a96-8466-4d36-a5b1-23cacfa6078b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.725399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1116/2229] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:34 2022] DELETE /identity/v3/projects/d640952ca4a14d9ab0c1fe397296ee5a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:34.730386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.730678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.730959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.735101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f0030c9d-59f8-4ab9-a563-a7d588bee003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f516632d25e443c68070f57bd3ca73a0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:34.751844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:34.753101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:34.763341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1114/2230] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:34 2022] DELETE /identity/v3/users/f516632d25e443c68070f57bd3ca73a0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:34.767262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:34.825650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1117/2231] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:34.833606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.833981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.834265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.852866 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:34.854091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:34.862397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1115/2232] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:34.867339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.877208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7978b68-69ca-44a4-b701-d330a2f5e36f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:34.882505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:34.883155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.883469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.883742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.884458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:34.884761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:34.886348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:34.903158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1118/2233] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:34.908182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.918433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841146-0dda-4b1c-9601-8982c8a84c5e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:34.923485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:34.924145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:34.924428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:34.924715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:34.926133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:34.928087 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:34.960715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1116/2234] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:34 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:34.965159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:34.975924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d5173-e505-4b6b-b3f4-60c0ec9c0603 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.054323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.063387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.064066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.064350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.064621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.065409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.067566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.067709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.082722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c55e1f-a7dc-4cb5-a95c-414cf191d60f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.088640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.100028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.100935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.101157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1119/2235] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:34 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:35.102082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.102567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] PATH_INFO: `/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.104674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b8f4fb1417e49e198f81c22fd9fbe5c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.111549 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.114988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c38c44-61ae-44c3-9e3b-3324e31504c2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.122314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.123042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.123365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.123640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.124509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.126128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.138599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1120/2236] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:35.143684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.155907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f62b2a-693a-486e-818a-db0b1cbda3d7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.161365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.162145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.162428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.163938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.164776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.167209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.183588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1121/2237] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:35.186334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.201863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774f149-adde-4f6a-9bad-da2d3aef16cc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.212063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b8f4fb1417e49e198f81c22fd9fbe5c', 'request_id': 'req-80f279eb-54cf-4985-b8ec-3497df85a83f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:35.220329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b8f4fb1417e49e198f81c22fd9fbe5c', 'request_id': 'req-80f279eb-54cf-4985-b8ec-3497df85a83f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:35.233046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b8f4fb1417e49e198f81c22fd9fbe5c', 'request_id': 'req-80f279eb-54cf-4985-b8ec-3497df85a83f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:35.249702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] The token cache is being invalidate because project 7b8f4fb1417e49e198f81c22fd9fbe5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:35.250294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:35.250999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-80f279eb-54cf-4985-b8ec-3497df85a83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:35.252913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1117/2238] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:35 2022] DELETE /identity/v3/projects/7b8f4fb1417e49e198f81c22fd9fbe5c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:35.259024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf6404-1493-4eb0-bd1c-6092b7ca9162 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.259335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf6404-1493-4eb0-bd1c-6092b7ca9162 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.259630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf6404-1493-4eb0-bd1c-6092b7ca9162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.277762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.279281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.279747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.280129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] PATH_INFO: `/v3/projects/94f88b74b9c64fc1a1e95bafec0e7e37/users/e9dea24d3b7a40feab8b3a1a7a6ec9ad/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.283386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-89bf6404-1493-4eb0-bd1c-6092b7ca9162 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:35.285635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89bf6404-1493-4eb0-bd1c-6092b7ca9162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.300579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f88b74b9c64fc1a1e95bafec0e7e37, user_id=e9dea24d3b7a40feab8b3a1a7a6ec9ad, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.304253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d19dc-9ee1-470d-b4da-0ff1c03d52d8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.304643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1118/2239] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:35.309060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:35.313826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.314330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.314625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.326635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1122/2240] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:35 2022] PUT /identity/v3/projects/94f88b74b9c64fc1a1e95bafec0e7e37/users/e9dea24d3b7a40feab8b3a1a7a6ec9ad/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:35.338990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.339458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.339905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.350505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:35.351844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.374260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] MFA Rules not processed for user `e9dea24d3b7a40feab8b3a1a7a6ec9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:35.389218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.399725 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:35.498883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1119/2241] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:35.504202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.512919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1123/2242] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:35.518403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b5888-f6b9-48a1-a3c7-0ed37de53cba None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.524392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.525320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.525320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.525712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.527493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:35.527607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.527687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.528534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.528704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.541153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1120/2243] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:35.545630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.546078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f496f-296e-4ef0-99d3-9cba01a2a093 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.551710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.553750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.554642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.555037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.556886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.557466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130fc5-d70e-48cd-bcaf-a88e21c3a88a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.559757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.564047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.564720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.565018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.565282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.566057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.568157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.592998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1124/2244] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:35.596900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.600660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1121/2245] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:35.601778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.672365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd8262-4c80-4562-959e-af33762da3f9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.678717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.679364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.679653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.679930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.680341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb52ef-6b98-492d-b28c-43a4846910b7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.680623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.682629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.685024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.685713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.686000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.686272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.686949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.688918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.708834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1122/2246] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:35.709910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.711934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1125/2247] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:35.715415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.722976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363776ad-248f-48a7-8537-03709cc85e39 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.725771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650587c-c948-4cb9-9d63-03158f48ab2c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.728246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.728987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.729376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.730860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.731557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.731840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.732062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.732162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.732827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.733366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.734452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.735464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.750903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1123/2248] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:35.751528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1126/2249] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:35.752511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.755920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.763395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870aed5-6f58-46bc-83a9-85f87aac933c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.768349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc1bb2-11c1-47c3-b926-31be5de3d632 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.774314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.774946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.774946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.775188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.775973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.776771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.777563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.779214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.779692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.780121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] PATH_INFO: `/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7/users/639b4ec1ced247619d70a2443859df6f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.790520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1127/2250] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:35.795090 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.801401 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=01564ebd67f342ee873bb4eb71d2b0e7, user_id=639b4ec1ced247619d70a2443859df6f, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.806931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.807882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1942ea-4e86-4d2b-97b8-8036c81bd130 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.814532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.815487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.815970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.816238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] PATH_INFO: `/v3/projects/bc53955c1b254e979a287932e5574bb9/users/86b52a28bedb4bb4964d81ef03145341/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.825986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1124/2251] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:35 2022] PUT /identity/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7/users/639b4ec1ced247619d70a2443859df6f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:35.830514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:35.838834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc53955c1b254e979a287932e5574bb9, user_id=86b52a28bedb4bb4964d81ef03145341, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.842545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff14de-b43a-47f6-a0bb-21781695fdcf admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.866809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1128/2252] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:35 2022] PUT /identity/v3/projects/bc53955c1b254e979a287932e5574bb9/users/86b52a28bedb4bb4964d81ef03145341/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:35.881613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.881907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.882195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.892130 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:35.896581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:35.912428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c489b-6ef5-43e2-bff2-5b61946f58ff admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:35.921470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] MFA Rules not processed for user `86b52a28bedb4bb4964d81ef03145341`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:35.923588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:35.983503 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:35.984184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:35.984468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:35.984767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:35.985496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:35.987158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:35.998061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1125/2253] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:35.998343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1129/2254] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:36.002224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.011155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.013055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.014832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b139b-abea-48dc-8498-00e9834d3884 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.020566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.021728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.022201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.022674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] PATH_INFO: `/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7/users/639b4ec1ced247619d70a2443859df6f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.024885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01564ebd67f342ee873bb4eb71d2b0e7, user_id=639b4ec1ced247619d70a2443859df6f, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.028071 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19107e40-cb80-47a4-a789-8f06ea9eff5c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.047299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1130/2255] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:36 2022] PUT /identity/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7/users/639b4ec1ced247619d70a2443859df6f/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:36.058472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.058798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.059567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.078566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b43f70-a008-434c-913a-0cd7f952f801 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.089821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] MFA Rules not processed for user `639b4ec1ced247619d70a2443859df6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:36.091168 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.137919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.138564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.139397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.139679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.170274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1131/2256] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:36.183333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.184851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.196949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2881ba8-e4f8-452d-b30b-c8cb2a2431ff None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.200820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.201597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.201909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.202120 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.213480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.214028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.215139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.215615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40002c44-6a0a-4f84-8c1c-c0113257aa91 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.220262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:36.225099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1126/2257] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:36.225758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1132/2258] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:36.320775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.322506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.408080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81dea7-51c1-4e23-811c-2ddf48e9361d service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.413966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.414864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.415206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.415506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] PATH_INFO: `/v3/users/ff75c0af77e04597a7b5746ccfbdac75` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.416711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff75c0af77e04597a7b5746ccfbdac75)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.419845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.563550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff75c0af77e04597a7b5746ccfbdac75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:36.566603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff75c0af77e04597a7b5746ccfbdac75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:36.579526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff75c0af77e04597a7b5746ccfbdac75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:36.580450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff75c0af77e04597a7b5746ccfbdac75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:36.611943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff75c0af77e04597a7b5746ccfbdac75'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:36.634392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1127/2259] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:36 2022] DELETE /identity/v3/users/ff75c0af77e04597a7b5746ccfbdac75 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:36.641293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.641764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.641868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.665945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:36.667183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.691226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.696813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.729989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:36.758260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1133/2260] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:36.764886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.766519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:36.776657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5eb1-1e8c-4379-84ae-fb810a3d7538 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.781297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639b4ec1ced247619d70a2443859df6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01564ebd67f342ee873bb4eb71d2b0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.781981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.782264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.782542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.783307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:36.783627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.785987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.797352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1128/2261] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:36.801610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.810996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8e86-0c14-47fc-96cb-334afe4ff453 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.816430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639b4ec1ced247619d70a2443859df6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01564ebd67f342ee873bb4eb71d2b0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.817079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.817365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.817643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.818328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.820274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.824566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dbc6c-d528-42ea-8807-af7ba7a582fa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.828145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.828785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.829068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.829345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.830909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:36.839965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.841719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.850343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1129/2262] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:36 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:36.852327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1134/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:36.854182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.917302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaeb2f-0b81-4a32-8ee2-c9576039ca3d tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:36.922375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639b4ec1ced247619d70a2443859df6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01564ebd67f342ee873bb4eb71d2b0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:36.923147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:36.923464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:36.923739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:36.924490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:36.924798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:36.927447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:36.940304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1130/2264] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:36.979879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:36.996988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ebc7b-c875-490f-8097-503d0b646d26 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.008696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '639b4ec1ced247619d70a2443859df6f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01564ebd67f342ee873bb4eb71d2b0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.010677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.011310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.012089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] PATH_INFO: `/v3/projects/7b4a4fe9507843feb694f7249765f8ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.013625 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7b4a4fe9507843feb694f7249765f8ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.029539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.084612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4a4fe9507843feb694f7249765f8ac', 'request_id': 'req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.091940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4a4fe9507843feb694f7249765f8ac', 'request_id': 'req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.099226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4a4fe9507843feb694f7249765f8ac', 'request_id': 'req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.114216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] The token cache is being invalidate because project 7b4a4fe9507843feb694f7249765f8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:37.114658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.115440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8c60e1ef-b1fd-41e5-97c6-6b746de4e230 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.117232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1135/2265] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:36 2022] DELETE /identity/v3/projects/7b4a4fe9507843feb694f7249765f8ac => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:37.150049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.152012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.248211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485db51f-3d48-46e7-8edd-c10ee817d158 tempest-VolumeQuotaClassesTest-1862952945 tempest-VolumeQuotaClassesTest-1862952945-project-admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.322765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.323429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.323922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.324229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] PATH_INFO: `/v3/projects/c04f22d5bad74880a321f1c0ffe2bd59` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.325511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04f22d5bad74880a321f1c0ffe2bd59)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.329022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.380138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:37.395124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04f22d5bad74880a321f1c0ffe2bd59', 'request_id': 'req-37b52e19-5c7a-4241-8689-ef29060663d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.403468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04f22d5bad74880a321f1c0ffe2bd59', 'request_id': 'req-37b52e19-5c7a-4241-8689-ef29060663d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.415807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04f22d5bad74880a321f1c0ffe2bd59', 'request_id': 'req-37b52e19-5c7a-4241-8689-ef29060663d6'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.428207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] The token cache is being invalidate because project c04f22d5bad74880a321f1c0ffe2bd59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:37.428484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.429341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.431322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1131/2266] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:37 2022] DELETE /identity/v3/projects/c04f22d5bad74880a321f1c0ffe2bd59 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:37.437113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e90f0-017a-4a4b-bb03-98859b8d64ed None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.437742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e90f0-017a-4a4b-bb03-98859b8d64ed None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.438327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e90f0-017a-4a4b-bb03-98859b8d64ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.439641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.441838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.467503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-646e90f0-017a-4a4b-bb03-98859b8d64ed None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:37.469119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-646e90f0-017a-4a4b-bb03-98859b8d64ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.526526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52e19-5c7a-4241-8689-ef29060663d6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.531589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.532767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.533051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.533321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] PATH_INFO: `/v3/users/e9dea24d3b7a40feab8b3a1a7a6ec9ad` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.535223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9dea24d3b7a40feab8b3a1a7a6ec9ad)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.541510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.557809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1136/2267] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:37.566812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.567219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.567595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.587209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:37.588404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.598796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1137/2268] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:37.602769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.611635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9dea24d3b7a40feab8b3a1a7a6ec9ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.612353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9dea24d3b7a40feab8b3a1a7a6ec9ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.620475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51b595-9491-4432-8530-92f28db744e7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.622301 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9dea24d3b7a40feab8b3a1a7a6ec9ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.623275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9dea24d3b7a40feab8b3a1a7a6ec9ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.626138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.626812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.627095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.627398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.628807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:37.629275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.631878 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.638296 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-19dfd0b0-3a64-44be-82f5-89b7c4de3c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9dea24d3b7a40feab8b3a1a7a6ec9ad'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:37.646340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1138/2269] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:20:37.651127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.653129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.662733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1132/2270] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:37 2022] DELETE /identity/v3/users/e9dea24d3b7a40feab8b3a1a7a6ec9ad => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:37.668667 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e52c9-3c6e-4bf2-8819-0d94b6fed3f6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.669545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:37.674441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.674757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.676952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.719984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:37.721717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.760444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.761409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.761805 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.762281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.763748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.767110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.788703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:37.824172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1139/2271] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:37 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:37.826482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.854246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1133/2272] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:37.865089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.866847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:37.899951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b804b9-c2bd-448d-a8a3-b57c60f0abf7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.907116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:37.907921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:37.908375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:37.908641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:37.910445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:37.918468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:37.957279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1140/2273] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:37 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:20:37.964587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:37.967534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc519e1-7968-424e-91ae-d54171f3f485 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:37.989706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86f330-cb62-4526-9105-3ef7b7639935 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.005671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.006914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.007435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.007924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.008852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.010841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.025278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1141/2274] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:38.029058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:38.043604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a18d5c-1898-4e13-b2c7-79504911611c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.049511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.050213 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.050488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.050950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.059041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.059776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.060061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.060343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.061077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.062775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.069636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.071084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a8fa3-fa6f-4d75-a19e-9f7cbb105477 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.076646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1142/2275] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:38.081928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:38.082895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1134/2276] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:20:38.093116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcd96f-cc73-41f1-93e8-c7482c0a3635 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.098979 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.100010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.100428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.100860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] PATH_INFO: `/v3/projects/ddf6034e4fe740b19807be75cb04bce5/users/2119337dc0cf4c5c9a11cf7e762fcd3c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.118828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf6034e4fe740b19807be75cb04bce5, user_id=2119337dc0cf4c5c9a11cf7e762fcd3c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.122062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.139834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1143/2277] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:38 2022] PUT /identity/v3/projects/ddf6034e4fe740b19807be75cb04bce5/users/2119337dc0cf4c5c9a11cf7e762fcd3c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:38.143422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.144738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.145064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.167997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:38.169940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:38.263504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1135/2278] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:38.270683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:38.273130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:38.315059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:38.360216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e414-5abb-41ca-8add-6fa4a5696be7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.365865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.367080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.368621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.369283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.380812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.382507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.393362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1144/2279] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:38.432116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:38.433715 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:38.459050 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a93c12-d4aa-4b03-8bde-5ac2332a6560 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.464880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.465737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.466017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.466286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] PATH_INFO: `/v3/projects/94f88b74b9c64fc1a1e95bafec0e7e37` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.467501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f88b74b9c64fc1a1e95bafec0e7e37)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.469528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.510692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:38.533822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f88b74b9c64fc1a1e95bafec0e7e37', 'request_id': 'req-b9170e54-9689-426a-a350-f685c67848f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.542585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f88b74b9c64fc1a1e95bafec0e7e37', 'request_id': 'req-b9170e54-9689-426a-a350-f685c67848f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.551585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f88b74b9c64fc1a1e95bafec0e7e37', 'request_id': 'req-b9170e54-9689-426a-a350-f685c67848f2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.579008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] The token cache is being invalidate because project 94f88b74b9c64fc1a1e95bafec0e7e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:38.579410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.580633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.582708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1136/2280] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:38 2022] DELETE /identity/v3/projects/94f88b74b9c64fc1a1e95bafec0e7e37 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:38.587013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:38.640311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97310fe-3f34-4bae-ba84-593707ad81a2 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:38.704866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:38.706001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:38.706925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:38.707348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] PATH_INFO: `/v3/users/639b4ec1ced247619d70a2443859df6f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:38.709069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639b4ec1ced247619d70a2443859df6f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:38.711161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:38.771977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639b4ec1ced247619d70a2443859df6f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.780231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639b4ec1ced247619d70a2443859df6f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.780933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639b4ec1ced247619d70a2443859df6f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.793112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639b4ec1ced247619d70a2443859df6f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.793909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-105b41e0-13cc-4548-86dd-21f9b1a0efe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639b4ec1ced247619d70a2443859df6f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:38.817221 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1145/2281] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:20:38 2022] DELETE /identity/v3/users/639b4ec1ced247619d70a2443859df6f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:20:39.420069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:39.428759 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:39.541944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9170e54-9689-426a-a350-f685c67848f2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:39.671542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:39.672733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:39.673578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:39.674009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] PATH_INFO: `/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:39.676729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=01564ebd67f342ee873bb4eb71d2b0e7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:39.679696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:39.751432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01564ebd67f342ee873bb4eb71d2b0e7', 'request_id': 'req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:39.761284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01564ebd67f342ee873bb4eb71d2b0e7', 'request_id': 'req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:39.772616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01564ebd67f342ee873bb4eb71d2b0e7', 'request_id': 'req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:39.790346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] The token cache is being invalidate because project 01564ebd67f342ee873bb4eb71d2b0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:20:39.791078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:39.792430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-6d6e0d9e-e45e-4c6c-8f22-3cd8cb02c5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:20:39.794940 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1137/2282] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:20:39 2022] DELETE /identity/v3/projects/01564ebd67f342ee873bb4eb71d2b0e7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:39.800316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:39.801650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:39.802372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:39.827030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:39.869781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:39.871721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:39.949245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1146/2283] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:39.957561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:39.957857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:39.958137 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:39.966011 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:39.985201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:39.986820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:39.996650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1138/2284] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:40.000784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.011204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ceef7-7ac9-433b-966b-dad9d5189eec None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.017444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.018205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.018587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.019679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.020539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:20:40.020908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.023402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.036020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1147/2285] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:20:40.040508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.051523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c8bc2-193f-441a-a7de-1400211e7f3b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.057435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.058331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.058790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.059267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.060711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.062773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.095756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1139/2286] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:40 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:40.097126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.108107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be54fd-91f8-4b28-b61b-6b65ceeabe56 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.176482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.177255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.177588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.177935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.178885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.180968 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.208455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1148/2287] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:40 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:40.209697 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.221917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947f3a-8962-48e2-a452-f40d8a806f51 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.230438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.231120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.231501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.232108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.232505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.234616 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.250798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1140/2288] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:40.252011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.263482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a0362-6185-491a-bfc2-0b7395f33d2b admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.270378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.271293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.271675 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.272057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.272945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.274811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a731c1-6294-4768-9f68-9e93104dbc45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.307143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1149/2289] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:40.308290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.324794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cce87-fd9f-4060-87f1-857d7f24f224 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.333497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.336363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.337487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.338046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] PATH_INFO: `/v3/projects/8a192348802d47f8b83b4626a06e65eb/users/b2d69fc2907d44269e6213dc929dd8a6/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.361794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a192348802d47f8b83b4626a06e65eb, user_id=b2d69fc2907d44269e6213dc929dd8a6, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.365817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.388390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1141/2290] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:40 2022] PUT /identity/v3/projects/8a192348802d47f8b83b4626a06e65eb/users/b2d69fc2907d44269e6213dc929dd8a6/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:40.399180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602857e7-36d3-415f-a7a4-23442be4a181 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.399547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602857e7-36d3-415f-a7a4-23442be4a181 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.399775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602857e7-36d3-415f-a7a4-23442be4a181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.450019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-602857e7-36d3-415f-a7a4-23442be4a181 None None] MFA Rules not processed for user `b2d69fc2907d44269e6213dc929dd8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:40.451713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-602857e7-36d3-415f-a7a4-23442be4a181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:40.519342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:40.552240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1150/2291] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:40.562895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:40.565580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:40.684277 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9039a4-56aa-4fd5-b606-45de332976d1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:40.688101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:40.688822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:40.689171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:40.689509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:40.699454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:40.701150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:40.713003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1142/2292] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:44.336691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.337273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.337371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.367036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] MFA Rules not processed for user `2119337dc0cf4c5c9a11cf7e762fcd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:44.368374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:44.448801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1151/2293] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:20:44.453312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.455625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:44.532279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176088c-afc9-414e-8db8-7e6546c5e09c service glance] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.538454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.539326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.539608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.539879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.540576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.542509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.582887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1143/2294] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:20:44 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:44.584061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.595397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72195941-5362-4eae-aaa0-7bbae24a3b64 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.658589 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.659351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.659634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.659908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.660644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.662578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.685582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1152/2295] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:20:44 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:20:44.689737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.700916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98ec3e-783d-4b17-a3fc-3b22601654f1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.705651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.706355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.706635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.706914 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.707678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.709298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.715552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:44.721173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1144/2296] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:20:44.725328 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.736122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0eba-d805-4cb7-9cb6-53704ea7b534 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.742129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.743041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.743419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.743689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] PATH_INFO: `/v3/projects/0b421c3d941742e7903fa2e8914d283b/users/573fc0ec10864754bab9f8a9a60693a5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.761479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b421c3d941742e7903fa2e8914d283b, user_id=573fc0ec10864754bab9f8a9a60693a5, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.764837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.782724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1153/2297] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:44 2022] PUT /identity/v3/projects/0b421c3d941742e7903fa2e8914d283b/users/573fc0ec10864754bab9f8a9a60693a5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:44.786463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.798106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81dd64-1b09-4de0-be6a-777d1966df20 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.872141 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.872919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.873233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.873521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.874249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.875918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580643e9-ea61-486d-b88e-95b8bb5a644e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.887275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1145/2298] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:20:44.891431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:44.907609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4005770-764e-4dad-9d45-cd5da40b51d5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:44.913563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:44.914488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:44.914751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:44.915324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] PATH_INFO: `/v3/projects/0b421c3d941742e7903fa2e8914d283b/users/573fc0ec10864754bab9f8a9a60693a5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:44.920258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b421c3d941742e7903fa2e8914d283b, user_id=573fc0ec10864754bab9f8a9a60693a5, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:44.920258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:44.935329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1154/2299] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:20:44 2022] PUT /identity/v3/projects/0b421c3d941742e7903fa2e8914d283b/users/573fc0ec10864754bab9f8a9a60693a5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:20:50.573558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6703d1-3c6d-4a17-adce-05eb3f04d5cd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:50.574107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6703d1-3c6d-4a17-adce-05eb3f04d5cd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:50.574303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6703d1-3c6d-4a17-adce-05eb3f04d5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:50.644970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ef6703d1-3c6d-4a17-adce-05eb3f04d5cd None None] MFA Rules not processed for user `573fc0ec10864754bab9f8a9a60693a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:50.646398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ef6703d1-3c6d-4a17-adce-05eb3f04d5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:50.816489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1146/2300] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:50.829042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:50.836696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:50.889941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:50.993713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8de80-0a71-4f7b-810a-974eb2c4d076 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:50.997572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:50.998242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:50.998522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:50.998795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:51.182436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:51.184439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:51.204961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1155/2301] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:20:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:20:51.427190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02591135-062d-404c-a328-cbb2b68995d6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:51.427543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02591135-062d-404c-a328-cbb2b68995d6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:51.427800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02591135-062d-404c-a328-cbb2b68995d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:51.446379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-02591135-062d-404c-a328-cbb2b68995d6 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:51.447639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-02591135-062d-404c-a328-cbb2b68995d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:51.508199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1147/2302] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:20:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:20:51.563498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:20:51.566048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:51.640024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741be9b-5c71-4f48-b903-f3b66a8a98a2 service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:20:51.645405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:20:51.646054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:51.646336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:51.646596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:51.655701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:20:51.657373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:20:51.669809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1156/2303] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:20:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:20:52.209145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfe6f-8a15-40d2-a3e3-d2dcacfc68cc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:20:52.209145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfe6f-8a15-40d2-a3e3-d2dcacfc68cc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:20:52.209145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cfe6f-8a15-40d2-a3e3-d2dcacfc68cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:20:52.235179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b07cfe6f-8a15-40d2-a3e3-d2dcacfc68cc None None] MFA Rules not processed for user `50d1dfc235cc4c5e8f375b4dbc1d0b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:20:52.251095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b07cfe6f-8a15-40d2-a3e3-d2dcacfc68cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:20:52.332263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:20:52.479149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1148/2304] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:20:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:01.663828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=67 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:02.453038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=13 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:02.601622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:02.677835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4705146-d2b5-4b23-ab15-e124a0d30e8b service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:02.684022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:02.685828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:02.686350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:02.686844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] PATH_INFO: `/v3/users/b2d69fc2907d44269e6213dc929dd8a6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:02.688519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d69fc2907d44269e6213dc929dd8a6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:02.691406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:02.749754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d69fc2907d44269e6213dc929dd8a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:02.764229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d69fc2907d44269e6213dc929dd8a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:02.764985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d69fc2907d44269e6213dc929dd8a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:02.782431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d69fc2907d44269e6213dc929dd8a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:02.783207 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d69fc2907d44269e6213dc929dd8a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:02.817599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1157/2305] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:02 2022] DELETE /identity/v3/users/b2d69fc2907d44269e6213dc929dd8a6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:02.822419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63de9-a1bb-4e5b-b564-dc7499b703bf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:02.822765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63de9-a1bb-4e5b-b564-dc7499b703bf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:02.823286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63de9-a1bb-4e5b-b564-dc7499b703bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:02.846289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-56f63de9-a1bb-4e5b-b564-dc7499b703bf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:02.847979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-56f63de9-a1bb-4e5b-b564-dc7499b703bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:02.919799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1149/2306] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:21:02.941418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:02.943376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:03.000576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb8ae-af41-4ce4-9cfc-5f17c52b9b99 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.000846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb8ae-af41-4ce4-9cfc-5f17c52b9b99 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.001202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb8ae-af41-4ce4-9cfc-5f17c52b9b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.025494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a99cb8ae-af41-4ce4-9cfc-5f17c52b9b99 None None] MFA Rules not processed for user `2119337dc0cf4c5c9a11cf7e762fcd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:03.026904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a99cb8ae-af41-4ce4-9cfc-5f17c52b9b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:03.042331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d2831-58ad-4bc6-9987-adb1a66fbe74 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.045990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.046642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.046923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.047223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.058659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.060177 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:03.070699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1158/2307] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:21:03.090548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1150/2308] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:03.450121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:03.458060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:03.468100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:03.479257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da45c88-0d98-480f-8791-8203bb9a4e7a service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.485971 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.487023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.487301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.487606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] PATH_INFO: `/v3/projects/8a192348802d47f8b83b4626a06e65eb` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.489002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a192348802d47f8b83b4626a06e65eb)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.498204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:03.554077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a192348802d47f8b83b4626a06e65eb', 'request_id': 'req-5403025f-ec9f-40ff-80ee-318183fd957e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:03.559813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a192348802d47f8b83b4626a06e65eb', 'request_id': 'req-5403025f-ec9f-40ff-80ee-318183fd957e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:03.566836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a192348802d47f8b83b4626a06e65eb', 'request_id': 'req-5403025f-ec9f-40ff-80ee-318183fd957e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:03.581115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] The token cache is being invalidate because project 8a192348802d47f8b83b4626a06e65eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:03.581469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:03.582128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5403025f-ec9f-40ff-80ee-318183fd957e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:03.588834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1159/2309] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:03 2022] DELETE /identity/v3/projects/8a192348802d47f8b83b4626a06e65eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:03.590698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.591194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.591337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.623657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:03.624972 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:03.690781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1151/2310] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:03.697808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.698302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.698406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.717505 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:03.718722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:03.728045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1160/2311] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:03.732420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:03.742787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cafd1-c779-4efd-8396-37f978e5b37b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.754184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.754947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.755993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.755993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.756765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:21:03.757139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.759078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:03.779276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1152/2312] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:21:03.783050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:03.798105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b781f52-b728-49ea-a67d-865326c48694 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.804387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.805063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.805357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.805663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.806359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.808684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:03.845595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1161/2313] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:21:03 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:03.849640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:03.861270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588057-5fec-4f6e-89d1-a46bdd4af508 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.894101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:03.934221 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.935044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.935432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.935781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.936482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.938429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:03.964523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1153/2314] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:21:03 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:03.967842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:03.982184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7703-8b7f-4d8d-8872-7888369ff417 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:03.987948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:03.988723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:03.989014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:03.989282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:03.990021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:03.991907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:04.009020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:04.011468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1162/2315] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:04.022820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e9bad-c83d-4dca-a856-a57d18c45189 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:04.036782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:04.037607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:04.037896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:04.038184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:04.038917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:04.040651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7bfdb-acbe-457c-9f65-e22d46fa6d04 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:04.058059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1154/2316] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:04.061647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:04.072087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325a30-0bff-4d9a-a3c2-82895066e257 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:04.076746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:04.077632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:04.077928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:04.078179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] PATH_INFO: `/v3/projects/9643d666032242bb882af509af881d39/users/91e581ee6ab9457db6e086ad5224210e/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:04.096710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9643d666032242bb882af509af881d39, user_id=91e581ee6ab9457db6e086ad5224210e, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:04.099792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:04.116460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1163/2317] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:04 2022] PUT /identity/v3/projects/9643d666032242bb882af509af881d39/users/91e581ee6ab9457db6e086ad5224210e/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:04.126560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00b41-c398-4619-a6d2-83062279c029 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:04.126851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00b41-c398-4619-a6d2-83062279c029 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:04.127169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00b41-c398-4619-a6d2-83062279c029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:04.161606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-79f00b41-c398-4619-a6d2-83062279c029 None None] MFA Rules not processed for user `91e581ee6ab9457db6e086ad5224210e`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:04.162971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-79f00b41-c398-4619-a6d2-83062279c029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:04.267179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1155/2318] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:04.279338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:04.281026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:04.342780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23e698-ff3b-4da7-8406-f435b2456a1d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:04.346692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:04.347370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:04.347656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:04.347932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:04.357058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:04.358563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:04.380121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1164/2319] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:05.331710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:05.333530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:05.334124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:05.368291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:05.372877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:05.529861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1156/2320] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:05.543727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:05.545368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:05.599356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:05.688338 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c316e-d0a6-4d8c-910f-228158ab3ca2 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:05.697496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:05.698604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:05.699017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:05.699749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:05.711436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:05.713700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b6bb6-15b8-4f61-901d-3cf15c40ab4d service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:05.729062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1165/2321] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:21:13.581177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:13.608104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:13.654126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee84933-c8a7-46f6-b798-09900879a13f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:13.659025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:13.659937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:13.660211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:13.660485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] PATH_INFO: `/v3/users/91e581ee6ab9457db6e086ad5224210e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:13.661688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e581ee6ab9457db6e086ad5224210e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:13.663704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:13.738619 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e581ee6ab9457db6e086ad5224210e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:13.739670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e581ee6ab9457db6e086ad5224210e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:13.748540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e581ee6ab9457db6e086ad5224210e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:13.749411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e581ee6ab9457db6e086ad5224210e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:13.767490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e581ee6ab9457db6e086ad5224210e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:13.803057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1157/2322] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:13 2022] DELETE /identity/v3/users/91e581ee6ab9457db6e086ad5224210e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:21:13.806429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:13.807098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:13.807788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:13.842200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:13.843634 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:13.958104 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1166/2323] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:13.966246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:13.968422 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:14.104664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe06c-152e-4139-bcbc-ae9fe17dea4f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:14.108815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:14.109706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:14.109992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:14.110272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:14.120652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:14.133105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d175e1-a6a4-4da6-9b8b-432c1efcdcaa service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:14.150980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1158/2324] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:14.662231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:14.663874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:14.698504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f447d23-c1fb-4f6e-bf21-57349101d427 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:14.724653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:14.726648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:14.727427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:14.728130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] PATH_INFO: `/v3/projects/9643d666032242bb882af509af881d39` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:14.729940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9643d666032242bb882af509af881d39)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:14.732981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:14.836311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9643d666032242bb882af509af881d39', 'request_id': 'req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:14.848049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9643d666032242bb882af509af881d39', 'request_id': 'req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:14.860639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9643d666032242bb882af509af881d39', 'request_id': 'req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:14.886642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] The token cache is being invalidate because project 9643d666032242bb882af509af881d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:14.886943 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:14.887756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-4f93be8e-f01d-4ae0-b67d-76f862cc72ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:14.889855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1167/2325] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:14 2022] DELETE /identity/v3/projects/9643d666032242bb882af509af881d39 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:14.896142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618963-e4cd-4884-958f-99e196168322 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:14.896707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618963-e4cd-4884-958f-99e196168322 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:14.896707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618963-e4cd-4884-958f-99e196168322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:14.936329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8d618963-e4cd-4884-958f-99e196168322 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:14.937577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d618963-e4cd-4884-958f-99e196168322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:14.990964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:15.018813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1159/2326] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:15.029145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.029442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.029727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.054309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:15.055711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:15.067280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1168/2327] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:15.069155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d618963-e4cd-4884-958f-99e196168322 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.078782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d618963-e4cd-4884-958f-99e196168322 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.084267 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.084953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.085219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.085525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.086251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:21:15.086636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.088880 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.103284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1160/2328] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:21:15.107297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.120473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215adb57-c492-4c2e-b631-deff3fcfcf63 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.125189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.125912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.126260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.127231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.128176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.130066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.165537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1169/2329] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:15.169574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.187068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995476b-2871-4942-a62e-87d02f7f87d6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.301290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.302654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.302951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.303310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.304300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.306244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.333349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1161/2330] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:15.335098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.353552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5870-536d-40d5-a390-78379ec9dc76 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.361491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.362110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.362396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.363142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.363523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.381171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.383657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:15.396865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1170/2331] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:15.401508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.411746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a72009-5072-415d-a4eb-8fa39acdd3eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.417110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.417736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.418017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.418292 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.419020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.420729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d939d-b9d7-4446-9858-1b1da0956430 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.442028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1162/2332] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:15.442128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.448244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672a38c-8b95-4403-89e4-3f6a4880839d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.453579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.454458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.455398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.455676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] PATH_INFO: `/v3/projects/c79b4c9f6fbd4304b9e9e789d7276b93/users/23a7950a933d4c57a582b40802799fc1/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.470995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79b4c9f6fbd4304b9e9e789d7276b93, user_id=23a7950a933d4c57a582b40802799fc1, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.473804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.487249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1171/2333] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:15 2022] PUT /identity/v3/projects/c79b4c9f6fbd4304b9e9e789d7276b93/users/23a7950a933d4c57a582b40802799fc1/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:15.497434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.497805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.499017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.528212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] MFA Rules not processed for user `23a7950a933d4c57a582b40802799fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:15.529530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:15.594305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1163/2334] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:15.599719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.663920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935cf4-04d8-4001-9eb8-a8599e4fd6e8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.668503 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.669152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.669437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.669710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.670400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.672335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.697691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1172/2335] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:15.701762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.713589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ce047-371e-46f3-9bc3-e0ac0b683a91 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.756111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:15.776926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.777596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.777882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.778163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.778864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.780834 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.801842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1164/2336] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:21:15 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:15.805639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.816529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439402a5-093f-491d-9959-726dc4265a62 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.821141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.821796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.822081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.822358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.823411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.825414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.837911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1173/2337] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:15.842061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.852275 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e7950-3443-43b7-bd49-ec0c9a9506e8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.857161 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.858044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.858324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.858596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] PATH_INFO: `/v3/projects/006a682de017451b9edbe50363d722be/users/03f239ad09894b33ab6df989df6e4dac/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.878891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=006a682de017451b9edbe50363d722be, user_id=03f239ad09894b33ab6df989df6e4dac, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.883198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:15.905116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1165/2338] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:15 2022] PUT /identity/v3/projects/006a682de017451b9edbe50363d722be/users/03f239ad09894b33ab6df989df6e4dac/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:21:15.905792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:15.916231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7004-9c90-4594-9437-5cb18eb08f74 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:15.989364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:15.993235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:15.993599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:15.994098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:15.995271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:15.998044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7e427-d27e-4377-be3a-5a1bc636b7ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:16.013241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1174/2339] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:21:16.017636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:16.028756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c052d6a-b01e-400b-bcd9-689839925956 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:16.034407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:16.035338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:16.035622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:16.035895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] PATH_INFO: `/v3/projects/006a682de017451b9edbe50363d722be/users/03f239ad09894b33ab6df989df6e4dac/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:16.037939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] RBAC: Authorizing `identity:create_grant(project_id=006a682de017451b9edbe50363d722be, user_id=03f239ad09894b33ab6df989df6e4dac, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:16.040659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:16.059529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1166/2340] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:16 2022] PUT /identity/v3/projects/006a682de017451b9edbe50363d722be/users/03f239ad09894b33ab6df989df6e4dac/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:16.070244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:16.070845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:16.071083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:16.102262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] MFA Rules not processed for user `03f239ad09894b33ab6df989df6e4dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:16.103557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:16.157474 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:16.207362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1175/2341] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:16.217151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:16.219261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:16.313027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6eb97-f2f1-4612-8f1d-c3ffba66100e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:16.318669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:16.319393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:16.319917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:16.320200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:16.408859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:16.408859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45757315-f891-45b3-bc2b-e31f422a745e service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:16.416955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1167/2342] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:19.270879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:19.388363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c834c-02e0-45cf-9f75-6637d48f93fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:19.396022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:19.396977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:19.397247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:19.397656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] PATH_INFO: `/v3/users/23a7950a933d4c57a582b40802799fc1` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:19.399174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a7950a933d4c57a582b40802799fc1)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:19.401403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:19.464010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a7950a933d4c57a582b40802799fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:19.472310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a7950a933d4c57a582b40802799fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:19.473012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a7950a933d4c57a582b40802799fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:19.481376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a7950a933d4c57a582b40802799fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:19.482212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a7950a933d4c57a582b40802799fc1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:19.499547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1176/2343] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:19 2022] DELETE /identity/v3/users/23a7950a933d4c57a582b40802799fc1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:19.505470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:19.505951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:19.506223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:19.525917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:19.527049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:19.549438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:19.591843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1168/2344] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:19.602142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:19.603823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:19.658536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:19.662205 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b1304-9156-4ac1-8303-f92b290cdaf6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:19.667089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:19.668330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:19.668779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:19.669060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:19.679442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:19.680902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:19.691137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1177/2345] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:19.962227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:19.963671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:19.975870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a588eb7-8010-4a26-a474-fb390c721d18 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:19.980561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:19.981607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:19.981954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:19.982323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] PATH_INFO: `/v3/projects/c79b4c9f6fbd4304b9e9e789d7276b93` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:19.983603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79b4c9f6fbd4304b9e9e789d7276b93)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:19.985596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.033983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79b4c9f6fbd4304b9e9e789d7276b93', 'request_id': 'req-f5a8ee3e-7632-4463-8201-cbc8037a8baa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.040089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79b4c9f6fbd4304b9e9e789d7276b93', 'request_id': 'req-f5a8ee3e-7632-4463-8201-cbc8037a8baa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.047181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79b4c9f6fbd4304b9e9e789d7276b93', 'request_id': 'req-f5a8ee3e-7632-4463-8201-cbc8037a8baa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.058035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] The token cache is being invalidate because project c79b4c9f6fbd4304b9e9e789d7276b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:20.058321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.058963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.060487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1169/2346] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:19 2022] DELETE /identity/v3/projects/c79b4c9f6fbd4304b9e9e789d7276b93 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:20.064526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:20.074053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771039-60eb-431b-ac13-5606f243e29f service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:20.100917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:20.102317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:20.133237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:20.134118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.134400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.134692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] PATH_INFO: `/v3/users/03f239ad09894b33ab6df989df6e4dac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.135875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f239ad09894b33ab6df989df6e4dac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:20.137814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.167724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ee3e-7632-4463-8201-cbc8037a8baa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:20.174170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:20.175164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.175477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.175747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] PATH_INFO: `/v3/users/86b52a28bedb4bb4964d81ef03145341` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.176959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b52a28bedb4bb4964d81ef03145341)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:20.178873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.188161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f239ad09894b33ab6df989df6e4dac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.195335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f239ad09894b33ab6df989df6e4dac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.195996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f239ad09894b33ab6df989df6e4dac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.203801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f239ad09894b33ab6df989df6e4dac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.204516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-3df87868-907b-421d-bc51-9ea6666061fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f239ad09894b33ab6df989df6e4dac'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.228690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1178/2347] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:20 2022] DELETE /identity/v3/users/03f239ad09894b33ab6df989df6e4dac => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:20.233412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b52a28bedb4bb4964d81ef03145341'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.234101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b52a28bedb4bb4964d81ef03145341'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.240510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:20.245320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b52a28bedb4bb4964d81ef03145341'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.248437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b52a28bedb4bb4964d81ef03145341'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.264138 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b52a28bedb4bb4964d81ef03145341'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.280324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1170/2348] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:20 2022] DELETE /identity/v3/users/86b52a28bedb4bb4964d81ef03145341 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:21:20.286504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.287015 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.287094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.305097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:20.306323 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:20.377300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1179/2349] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:20.387080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:20.388527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:20.463942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbed856-660a-4d77-8667-283432bde3f6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:20.467854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:20.468430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.468709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.468980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.479740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:20.481306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.496304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1171/2350] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:20.877458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:20.878900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:20.891815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:20.894350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:20.906999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd1bce-4160-4d01-8ce6-4f1e26d5f261 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:20.912433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:20.913303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.913610 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.913849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] PATH_INFO: `/v3/projects/bc53955c1b254e979a287932e5574bb9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.915173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc53955c1b254e979a287932e5574bb9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:20.917461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.924549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:20.948563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39b0ba-5515-4b1c-8a34-355cf3c4eb42 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:20.959695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:20.961769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:20.962468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:20.963224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] PATH_INFO: `/v3/projects/006a682de017451b9edbe50363d722be` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:20.964710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006a682de017451b9edbe50363d722be)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:20.968014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:20.982156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc53955c1b254e979a287932e5574bb9', 'request_id': 'req-1e857281-44d0-44d2-8ecb-1395461ab03c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.988650 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc53955c1b254e979a287932e5574bb9', 'request_id': 'req-1e857281-44d0-44d2-8ecb-1395461ab03c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:20.995021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc53955c1b254e979a287932e5574bb9', 'request_id': 'req-1e857281-44d0-44d2-8ecb-1395461ab03c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.011414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] The token cache is being invalidate because project bc53955c1b254e979a287932e5574bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:21.011931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.012893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1e857281-44d0-44d2-8ecb-1395461ab03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.014750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1180/2351] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:20 2022] DELETE /identity/v3/projects/bc53955c1b254e979a287932e5574bb9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:21.027970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006a682de017451b9edbe50363d722be', 'request_id': 'req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.037654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006a682de017451b9edbe50363d722be', 'request_id': 'req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.050358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006a682de017451b9edbe50363d722be', 'request_id': 'req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.063596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] The token cache is being invalidate because project 006a682de017451b9edbe50363d722be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:21.064475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.065413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4175fa14-53e3-4e83-96e6-53a8cb5c6a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:21.070339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1172/2352] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:20 2022] DELETE /identity/v3/projects/006a682de017451b9edbe50363d722be => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:21.072202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692597e-86be-4064-bb97-a06458b02113 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.072563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692597e-86be-4064-bb97-a06458b02113 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.072915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692597e-86be-4064-bb97-a06458b02113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.092981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d692597e-86be-4064-bb97-a06458b02113 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:21.094312 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d692597e-86be-4064-bb97-a06458b02113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:21.160607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1181/2353] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:21.169930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.170209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.170480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.190287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:21.191626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:21.200351 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1173/2354] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:21.204176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692597e-86be-4064-bb97-a06458b02113 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.216187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692597e-86be-4064-bb97-a06458b02113 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.222397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.222991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.223363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.223642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.224368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:21:21.224672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.226302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.238210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1182/2355] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:21:21.242531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.251807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5a9e7-4874-48b3-865c-047fec4df0d8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.256387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.257022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.257299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.257568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.258257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.260209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.286014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1174/2356] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:21:21 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:21.289941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.300045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce364b5-02b5-4f6c-9ce1-7185e6b5745f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.355156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.355850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.356135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.356412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.357130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.359036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.383510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1183/2357] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:21:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:21:21.384913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.397269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5f1b2-e7c3-44bf-9a70-6b506b0bef3e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.402079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.402740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.403021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.403322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.404048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.405658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.417182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1175/2358] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:21.421157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.431261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48124-6de8-43d3-b143-5fd79c17499d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.435912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.436562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.436849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.437127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.437847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.439723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b432f78-cebc-428a-bd70-0853edb96ee6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.444851 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:21.451004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1184/2359] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:21.455143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.465029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c9991-2697-4e27-a6bf-b72b76b163c0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.469632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.471084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.471504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.471785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] PATH_INFO: `/v3/projects/eb5c5a3c3f3d477084f2a60bcd8f11cf/users/66e85fa33e3e43d5a1964bfba622e857/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.483872 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:21.488000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5c5a3c3f3d477084f2a60bcd8f11cf, user_id=66e85fa33e3e43d5a1964bfba622e857, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.491353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.504511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1176/2360] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:21 2022] PUT /identity/v3/projects/eb5c5a3c3f3d477084f2a60bcd8f11cf/users/66e85fa33e3e43d5a1964bfba622e857/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:21.514808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1248-ef77-41aa-a23b-33b591ec0b3d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.515105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1248-ef77-41aa-a23b-33b591ec0b3d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.515425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c1248-ef77-41aa-a23b-33b591ec0b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.544492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d91c1248-ef77-41aa-a23b-33b591ec0b3d None None] MFA Rules not processed for user `66e85fa33e3e43d5a1964bfba622e857`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:21.545946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d91c1248-ef77-41aa-a23b-33b591ec0b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:21.620299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1185/2361] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:21:21.631737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:21.633475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:21.693093 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eff8-1961-4383-b4bd-b6ce3cf0694d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:21.696745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:21.697383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:21.697663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:21.697933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:21.708060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:21.709507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:21.718695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1177/2362] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:22.117737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01f7b-98aa-48b2-b22b-7a9c943723f9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:22.118343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01f7b-98aa-48b2-b22b-7a9c943723f9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:22.118841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e01f7b-98aa-48b2-b22b-7a9c943723f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:22.147362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d9e01f7b-98aa-48b2-b22b-7a9c943723f9 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:22.147362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d9e01f7b-98aa-48b2-b22b-7a9c943723f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:22.225941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1186/2363] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:22.244532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:22.246178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:22.346042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877430fa-70c5-41f4-a95f-c81f18921c35 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:22.351584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:22.352332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:22.353175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:22.353529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:22.373043 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:22.374767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:22.383994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1178/2364] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:21:29.910737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:29.911707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:29.911707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:30.026217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:30.027731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:30.056025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1187/2365] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:21:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:30.074937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:30.131724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e133bc-6a27-4356-94f8-9fa58739c050 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:30.143728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:30.147813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:30.148294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:30.148767 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:30.170022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:30.174649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab5843-7d58-4f52-a43e-8b45bfb0b85d service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:30.191321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1179/2366] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:21:33.209106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:33.364129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:33.384701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00230c-bf2d-423a-8c31-d1f19257d7a3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:33.413620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:33.414075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:33.414347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:33.414621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] PATH_INFO: `/v3/users/66e85fa33e3e43d5a1964bfba622e857` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:33.415981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] RBAC: Authorizing `identity:delete_user(user_id=66e85fa33e3e43d5a1964bfba622e857)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:33.425111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:33.560627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66e85fa33e3e43d5a1964bfba622e857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:33.580772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66e85fa33e3e43d5a1964bfba622e857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:33.581570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66e85fa33e3e43d5a1964bfba622e857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:33.611893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66e85fa33e3e43d5a1964bfba622e857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:33.612545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66e85fa33e3e43d5a1964bfba622e857'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:33.657731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1188/2367] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:33 2022] DELETE /identity/v3/users/66e85fa33e3e43d5a1964bfba622e857 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:33.662070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:33.662643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:33.662643 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:33.729631 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:33.737895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:34.016388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1180/2368] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:34.026812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:34.028393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:34.141208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9c8e4-aebe-456e-9483-047e2fb5203d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:34.154919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:34.155700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:34.155930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:34.156470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:34.171024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:34.172798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e4efd-282c-400d-971e-14e8a66b498f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:34.200822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1189/2369] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:34.610131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:34.611930 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:34.619833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:34.630091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7f5a2-7a7e-441e-a954-534a891c0d09 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:34.667108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:34.670064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:34.670592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:34.671124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] PATH_INFO: `/v3/projects/eb5c5a3c3f3d477084f2a60bcd8f11cf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:34.679601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5c5a3c3f3d477084f2a60bcd8f11cf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:34.681949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:34.882434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5c5a3c3f3d477084f2a60bcd8f11cf', 'request_id': 'req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:34.910151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5c5a3c3f3d477084f2a60bcd8f11cf', 'request_id': 'req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:34.933341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5c5a3c3f3d477084f2a60bcd8f11cf', 'request_id': 'req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:34.953782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] The token cache is being invalidate because project eb5c5a3c3f3d477084f2a60bcd8f11cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:34.954315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:34.955864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a83beb36-7c9a-446d-8f2c-bde3249ee4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:34.957933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1181/2370] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:34 2022] DELETE /identity/v3/projects/eb5c5a3c3f3d477084f2a60bcd8f11cf => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:40.158868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.158868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.160337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.161423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1190/2371] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:21:40.167642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61590ac-c3db-4718-a87a-735bb73f948b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.168175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61590ac-c3db-4718-a87a-735bb73f948b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.168440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61590ac-c3db-4718-a87a-735bb73f948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.237689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e61590ac-c3db-4718-a87a-735bb73f948b None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:40.239455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e61590ac-c3db-4718-a87a-735bb73f948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:40.348344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1182/2372] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:40.377164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:40.378701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:40.482083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6bec3-f785-4b32-8892-84be8ebb3ec8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:40.487377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:40.488347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.488750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.489199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.502383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:40.504196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd13e20-e8ce-4bbb-ae01-06bfd221a97a service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:40.511733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:40.519662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1191/2373] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:40.602158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.602430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.602704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.604127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1183/2374] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:21:40.609954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249532a7-6efd-40d1-9ca7-3c42e98db460 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.610870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249532a7-6efd-40d1-9ca7-3c42e98db460 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.611222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249532a7-6efd-40d1-9ca7-3c42e98db460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.633403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-249532a7-6efd-40d1-9ca7-3c42e98db460 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:40.634741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-249532a7-6efd-40d1-9ca7-3c42e98db460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:40.696602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1192/2375] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:40.712576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:40.715040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:40.730334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44419ffc-8276-4fef-b7d3-35a829063230 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:40.734860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:40.735963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.736386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.736779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.746320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:40.748335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e54218-e070-433d-82a5-ed5d17ea7127 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:40.758553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1184/2376] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:21:40.857574 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaff53-bca8-4acb-980a-faca7da324bb None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.858133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaff53-bca8-4acb-980a-faca7da324bb None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.858524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaff53-bca8-4acb-980a-faca7da324bb None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.860180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1193/2377] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:21:40.866125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.866500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.866852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.887711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:40.889407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:40.900418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1185/2378] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:21:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:40.912268 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aaff53-bca8-4acb-980a-faca7da324bb None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:40.926390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aaff53-bca8-4acb-980a-faca7da324bb None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:40.930047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:40.930712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:40.931004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:40.931321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:40.942838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:40.944495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a53f65-b77f-479d-adac-4745f248c29d service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:40.958641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1194/2379] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:43.661657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:43.663711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:43.693365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b24d7-857a-4357-a880-5b0ffa95416b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:43.702670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:43.703564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:43.704950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:43.705460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:43.708201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:43.743032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:43.820895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:43.822538 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:43.841891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1186/2380] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:45.079636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54830-f259-48ad-9f89-c544e0587877 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:45.080663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54830-f259-48ad-9f89-c544e0587877 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:45.081144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54830-f259-48ad-9f89-c544e0587877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:45.122960 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7ad54830-f259-48ad-9f89-c544e0587877 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:45.125034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7ad54830-f259-48ad-9f89-c544e0587877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:45.211381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1195/2381] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:21:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:45.227033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:45.229057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:45.364946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f57e2c-6157-4141-b971-82924e0c0b56 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:45.371039 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:45.371831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:45.372081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:45.372362 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:45.382338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:45.386537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:45.399188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1187/2382] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:21:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:21:50.568506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835eb8-60d0-4478-aacd-333fbfc6b6e8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:50.569103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835eb8-60d0-4478-aacd-333fbfc6b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:50.569103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835eb8-60d0-4478-aacd-333fbfc6b6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:50.613890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-de835eb8-60d0-4478-aacd-333fbfc6b6e8 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:50.615335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-de835eb8-60d0-4478-aacd-333fbfc6b6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:50.638685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1196/2383] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:21:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:50.663548 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:50.676194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0a35-2fd3-4596-89c8-c5c8b4f21aed service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:50.682423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:50.683147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:50.683516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:50.683790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:50.692734 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:50.694327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f606aca-a36c-432d-ac36-5503ee3299e1 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:50.702588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1188/2384] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:21:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:21:52.991779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:52.993055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:52.993527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:52.995611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1197/2385] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:21:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:21:53.006324 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.007220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.007782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.076560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] MFA Rules not processed for user `c33cd0ddf4fc49e0b8e9d6f0863f81dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:53.076560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:53.100767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:53.104033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:53.176666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:53.188138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47da06-2aa6-4fdc-b2af-114ed865a4cc None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:53.196606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:53.197511 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.197802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.205179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1189/2386] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:53.205925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] PATH_INFO: `/v3/users/50d1dfc235cc4c5e8f375b4dbc1d0b51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.207481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d1dfc235cc4c5e8f375b4dbc1d0b51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:53.209465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:53.217355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:53.218939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:53.233583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31229a56-b067-46ff-8258-7a72f2a81122 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:53.239016 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:53.239717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.240118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.240269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.260480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:53.262222 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:53.266708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:53.275000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1190/2387] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:21:53.278755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d1dfc235cc4c5e8f375b4dbc1d0b51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.287005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d1dfc235cc4c5e8f375b4dbc1d0b51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.287756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d1dfc235cc4c5e8f375b4dbc1d0b51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.295784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d1dfc235cc4c5e8f375b4dbc1d0b51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.296691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-395df409-7d0c-45e1-bc79-f6cfa82320c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d1dfc235cc4c5e8f375b4dbc1d0b51'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.336083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1198/2388] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:21:53 2022] DELETE /identity/v3/users/50d1dfc235cc4c5e8f375b4dbc1d0b51 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:53.619639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:53.632925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e43067-cf30-4a14-ba59-e70350a8ce3c service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:53.691150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:53.692048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.692329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.692601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] PATH_INFO: `/v3/projects/5b8a415a2a93492fa4f6b3c9acc0b080` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.693780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8a415a2a93492fa4f6b3c9acc0b080)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:53.695740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:53.747181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8a415a2a93492fa4f6b3c9acc0b080', 'request_id': 'req-a380b63c-2a10-45eb-b03e-66aa597aaa4e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.754195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8a415a2a93492fa4f6b3c9acc0b080', 'request_id': 'req-a380b63c-2a10-45eb-b03e-66aa597aaa4e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.764857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8a415a2a93492fa4f6b3c9acc0b080', 'request_id': 'req-a380b63c-2a10-45eb-b03e-66aa597aaa4e'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.778408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] The token cache is being invalidate because project 5b8a415a2a93492fa4f6b3c9acc0b080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:21:53.778760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.779398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a380b63c-2a10-45eb-b03e-66aa597aaa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:21:53.781031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1191/2389] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:21:53 2022] DELETE /identity/v3/projects/5b8a415a2a93492fa4f6b3c9acc0b080 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:53.787616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.787910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.788191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.807757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:53.808972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:53.869398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1199/2390] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:53.877778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.878155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.878511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.903720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:53.905239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:53.924595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1192/2391] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:53.928856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:53.939395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b28497-79d6-4232-959f-a082a43b71c3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:53.947481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:53.948163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.948446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.948717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.949448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:21:53.949754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:53.951402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:53.964255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1200/2392] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:21:53.968546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:53.977808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfeba4-7dd9-49dd-ae29-dddd28957c9e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:53.982427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:53.983069 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:53.983381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:53.983656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:53.984349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:53.986416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.012397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1193/2393] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:21:53 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:54.017243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:54.027232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586af3-1452-4fb9-9ef2-a53830655982 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:54.088684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:54.089373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.089655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.089931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.090647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:54.092631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.117469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1201/2394] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:21:54 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:21:54.121515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:54.132805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5962ae-fe0f-4a2a-ba3a-7b0d904a6f80 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:54.137496 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:54.138219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.138566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.139046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.140047 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:54.142812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.153567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1194/2395] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:21:54.157631 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:54.167708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bb70-ccdd-4219-8655-6d38cdf5a6e1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:54.172919 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:54.173759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.174041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.174319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.175067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:54.176691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ed77-f208-4989-a91d-7466cdef5a0e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.187229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1202/2396] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:21:54.192142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:54.202563 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1fb67a-494b-43ba-992a-2340555e78fe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:54.207394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:54.208338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.208732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.209534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] PATH_INFO: `/v3/projects/326f6a78057d4f92a3bbb2e52802f904/users/455cf23c4c4d4254ad7f1121f2f02c7e/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.225026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=326f6a78057d4f92a3bbb2e52802f904, user_id=455cf23c4c4d4254ad7f1121f2f02c7e, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:54.228001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.238031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:54.247278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1195/2397] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:21:54 2022] PUT /identity/v3/projects/326f6a78057d4f92a3bbb2e52802f904/users/455cf23c4c4d4254ad7f1121f2f02c7e/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:21:54.257316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d651-0b75-4868-be11-90210dd53c38 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.257754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d651-0b75-4868-be11-90210dd53c38 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.258069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d651-0b75-4868-be11-90210dd53c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.295464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c2b7d651-0b75-4868-be11-90210dd53c38 None None] MFA Rules not processed for user `455cf23c4c4d4254ad7f1121f2f02c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:54.296765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c2b7d651-0b75-4868-be11-90210dd53c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:54.306568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:21:54.370479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1203/2398] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:21:54.381377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:21:54.383072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:54.447103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bc003-42f0-410a-9a60-b0a47dc8b287 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:21:54.451232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:21:54.451983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:54.452335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:54.452680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:54.463438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:21:54.465271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed3d50-6db1-4b31-b720-005993fd7843 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:21:54.482890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1196/2399] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:21:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:21:55.901148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:21:55.901710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:21:55.901710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:21:55.919981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] MFA Rules not processed for user `455cf23c4c4d4254ad7f1121f2f02c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:21:55.921287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:21:55.928815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1204/2400] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:21:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:04.472023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=26 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:05.920497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=26 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:11.666501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:11.667022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:11.667022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:11.684394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] MFA Rules not processed for user `7708b4816bcf4301ba7cc826d533aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:11.685574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:11.740102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1197/2401] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:13.321677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:13.323155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:13.375823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:13.378189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:13.423569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4e3a5-00e8-4f41-923c-df5e43560c17 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:13.429592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:13.430739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:13.431334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:13.431879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:13.434048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:13.450093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:13.452865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086792c-99c7-4911-ad7e-bb8d402e8501 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:13.456974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dadd5f-113b-40c8-bf78-a474d43341e5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:13.462131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:13.463024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:13.463544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:13.463828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:13.470964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1205/2402] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:22:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:22:13.473674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:13.475309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:13.489966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1198/2403] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:22:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:22:14.213839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff7e0-e2ed-46b8-bde7-a1ba2942d593 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:14.214128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff7e0-e2ed-46b8-bde7-a1ba2942d593 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:14.214405 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff7e0-e2ed-46b8-bde7-a1ba2942d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:14.242343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e95ff7e0-e2ed-46b8-bde7-a1ba2942d593 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:14.242343 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e95ff7e0-e2ed-46b8-bde7-a1ba2942d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:14.301219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1206/2404] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:14.323059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:14.324519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:14.395672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44875b44-0ba8-4cbc-ac6b-c0732bc0f9e2 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:14.400655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:14.401303 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:14.401580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:14.401850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:14.410484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:14.412104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:14.420273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1199/2405] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:22:16.035038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82bb8-bf25-4df0-91fb-dc24bc41c02a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:16.035616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82bb8-bf25-4df0-91fb-dc24bc41c02a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:16.035616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82bb8-bf25-4df0-91fb-dc24bc41c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:16.056813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-93c82bb8-bf25-4df0-91fb-dc24bc41c02a None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:16.058230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-93c82bb8-bf25-4df0-91fb-dc24bc41c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:16.069078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1207/2406] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:16.084254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:16.097608 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96ce28-fdd2-4dd1-8ce9-bf36d008116e service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:16.104327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:16.105298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:16.105682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:16.106053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:16.117609 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:16.119689 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:16.129318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1200/2407] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 03:22:16.739161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaa98-df87-415f-9411-0a4e15c24bbd None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:16.739777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaa98-df87-415f-9411-0a4e15c24bbd None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:16.740157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aaa98-df87-415f-9411-0a4e15c24bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:16.783679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-887aaa98-df87-415f-9411-0a4e15c24bbd None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:16.785142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-887aaa98-df87-415f-9411-0a4e15c24bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:16.793666 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1208/2408] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:16.809965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:16.822017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0be0d-3f0a-402e-991a-4c6a1b365d06 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:16.827897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:16.829471 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:16.830106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:16.830607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:16.842455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:16.844185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:16.848501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:16.855376 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1201/2409] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:22:17.064663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ae12-b9d4-4fe4-8dd5-bd127c2f4fbf None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:17.065145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ae12-b9d4-4fe4-8dd5-bd127c2f4fbf None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:17.065237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283ae12-b9d4-4fe4-8dd5-bd127c2f4fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:17.084216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5283ae12-b9d4-4fe4-8dd5-bd127c2f4fbf None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:17.085414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5283ae12-b9d4-4fe4-8dd5-bd127c2f4fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:17.092705 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1209/2410] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:22:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:17.105132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:17.115926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182c7aa-4e80-4d91-aee8-8610ba68ad37 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:17.120915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:17.121551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:17.121826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:17.122096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:17.131223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:17.132847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:17.141293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1202/2411] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:22:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:22:17.464506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:17.464798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:17.465084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:17.483526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:17.485304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:17.498825 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1210/2412] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:22:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:17.513154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:17.524027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564ca8-746e-49c7-bc69-ae5681e7b1f3 service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:17.528970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:17.529599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:17.529884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:17.530162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:17.538181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:17.539838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e22e3-78cc-4533-8f89-8e685205b90b service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:17.548467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1203/2413] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:22:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:22:19.643537 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:19.711107 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e6c28-6b9b-4cd8-9c93-c971f0a5a195 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:19.716172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:19.716890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:19.717173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:19.717449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] PATH_INFO: `/v3/users/455cf23c4c4d4254ad7f1121f2f02c7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:19.718648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455cf23c4c4d4254ad7f1121f2f02c7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:19.720680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:19.732726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:19.789782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455cf23c4c4d4254ad7f1121f2f02c7e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:19.802685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455cf23c4c4d4254ad7f1121f2f02c7e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:19.804190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455cf23c4c4d4254ad7f1121f2f02c7e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:19.813918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455cf23c4c4d4254ad7f1121f2f02c7e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:19.814557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455cf23c4c4d4254ad7f1121f2f02c7e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:19.838607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1211/2414] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:22:19 2022] DELETE /identity/v3/users/455cf23c4c4d4254ad7f1121f2f02c7e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:22:19.843036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:19.844526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:19.844526 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:19.869974 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:19.871699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:19.950461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1204/2415] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:22:19.961880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:19.963984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.025600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd551db-aeb1-4596-bafd-e35df87cfe17 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.029283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.029933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.030218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.030526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.039786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.041231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304409b-0068-4ab6-b966-1937899042ee service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.051402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1212/2416] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:22:20.232976 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.234410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.246478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f02d0c-1b10-426a-b4ed-5042c3386fec None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.252179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.253032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.253306 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.253573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] PATH_INFO: `/v3/projects/326f6a78057d4f92a3bbb2e52802f904` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.254765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326f6a78057d4f92a3bbb2e52802f904)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.256733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.317465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326f6a78057d4f92a3bbb2e52802f904', 'request_id': 'req-11049f2d-af59-453c-9fff-d50676489934'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:20.323724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326f6a78057d4f92a3bbb2e52802f904', 'request_id': 'req-11049f2d-af59-453c-9fff-d50676489934'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:20.332431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326f6a78057d4f92a3bbb2e52802f904', 'request_id': 'req-11049f2d-af59-453c-9fff-d50676489934'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:20.342923 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] The token cache is being invalidate because project 326f6a78057d4f92a3bbb2e52802f904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:22:20.343209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:20.343951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-11049f2d-af59-453c-9fff-d50676489934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:22:20.345673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1205/2417] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:22:20 2022] DELETE /identity/v3/projects/326f6a78057d4f92a3bbb2e52802f904 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:22:20.351028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.351417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.351694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.374457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:20.374457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.434010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1213/2418] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:20.442434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfed07-e868-4827-8542-389f1180e581 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.442722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfed07-e868-4827-8542-389f1180e581 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.442997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfed07-e868-4827-8542-389f1180e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.461702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-49cfed07-e868-4827-8542-389f1180e581 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:20.462924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-49cfed07-e868-4827-8542-389f1180e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.471094 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1206/2419] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:20.477219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.489797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5482d7a-fb36-49d8-a815-48d7e0242c21 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.494433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.495087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.495400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.495678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.496399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:22:20.496707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.498305 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.511001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1214/2420] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:22:20.515129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfed07-e868-4827-8542-389f1180e581 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.525088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfed07-e868-4827-8542-389f1180e581 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.531625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.532290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.532571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.532842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.533537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.535485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.564014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1207/2421] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:22:20 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:22:20.569996 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.581130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3a17-ac66-4d59-a1c1-b7c97dda5f2c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.594113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:20.643529 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.644181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.644472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.644749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.645430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.647378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.667331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1215/2422] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:22:20 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:22:20.671365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.682001 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eea62f-cda6-4255-8916-7a2274feee3b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.686593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.687263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.687655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.687655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.688592 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.690208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.700754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1208/2423] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:22:20.705490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.715456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128d870-63e9-4f05-b992-43a8d5be81b8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.720633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.721288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.721572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.721846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.722567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.724214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006b49-e1b8-4829-87f9-5a21c6359d6c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.734126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1216/2424] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:22:20.738299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.748197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c07e1b-6ee8-4c9c-82c5-e7a7f851633a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.753368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.754233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.754510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.754777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] PATH_INFO: `/v3/projects/b14b528255fe4423b6129876f7cf471e/users/1009f79ba74e44a1be8781d74e95baa6/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.767846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:20.771225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14b528255fe4423b6129876f7cf471e, user_id=1009f79ba74e44a1be8781d74e95baa6, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.774787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.791512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1209/2425] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:22:20 2022] PUT /identity/v3/projects/b14b528255fe4423b6129876f7cf471e/users/1009f79ba74e44a1be8781d74e95baa6/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:22:20.796732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07511-7463-4a28-8d8f-1457844bc661 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.797056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07511-7463-4a28-8d8f-1457844bc661 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.797443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d07511-7463-4a28-8d8f-1457844bc661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.815647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b3d07511-7463-4a28-8d8f-1457844bc661 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:20.816848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b3d07511-7463-4a28-8d8f-1457844bc661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.874813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1217/2426] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:20.885020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:20.886944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:20.953167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19cad-790f-47ab-9368-a86ff49f8207 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:20.957566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:20.958220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:20.958509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:20.958782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:20.968947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:20.970440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:20.982912 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1210/2427] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:22:21.769182 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001bf-ffd2-4c0a-972a-7600f064d149 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:21.769686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001bf-ffd2-4c0a-972a-7600f064d149 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:21.769809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d001bf-ffd2-4c0a-972a-7600f064d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:21.797310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-88d001bf-ffd2-4c0a-972a-7600f064d149 None None] MFA Rules not processed for user `1009f79ba74e44a1be8781d74e95baa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:21.798484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-88d001bf-ffd2-4c0a-972a-7600f064d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:21.864724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1218/2428] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:22:21.876337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:22:21.877728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:21.936154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4d7ff-fa1c-40fc-aa61-cb8c9b9c75d9 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:22:21.940052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:22:21.940696 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:21.940977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:21.941253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:21.949908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:22:21.951370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306822af-0039-4ece-b7fc-b3236a4569cc service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:22:21.960831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1211/2429] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:22:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:22:23.317907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bd0c7-379a-4860-beda-f6257933b606 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:22:23.318414 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bd0c7-379a-4860-beda-f6257933b606 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:22:23.318510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bd0c7-379a-4860-beda-f6257933b606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:22:23.336937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-897bd0c7-379a-4860-beda-f6257933b606 None None] MFA Rules not processed for user `1009f79ba74e44a1be8781d74e95baa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:22:23.338187 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-897bd0c7-379a-4860-beda-f6257933b606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:22:23.346466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1219/2430] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:22:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:22:31.956889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:22:33.331316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:23:18.390302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56bbc-e8c3-48ab-9dc6-80ddecf9be40 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:18.390823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56bbc-e8c3-48ab-9dc6-80ddecf9be40 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:18.390997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56bbc-e8c3-48ab-9dc6-80ddecf9be40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:18.409559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a0b56bbc-e8c3-48ab-9dc6-80ddecf9be40 None None] MFA Rules not processed for user `3cf837e2ab5b4e58844a1023cf5f9237`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:23:18.410825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a0b56bbc-e8c3-48ab-9dc6-80ddecf9be40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:18.468480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1212/2431] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:23:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:23:28.460884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=25 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:23:29.000273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:29.000765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:29.001419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:29.020229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] MFA Rules not processed for user `1009f79ba74e44a1be8781d74e95baa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:23:29.021412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:29.028106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1220/2432] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:23:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:23:39.013807 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:23:43.734764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:43.735207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:43.735411 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:43.753307 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:23:43.754501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:43.811208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1213/2433] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:23:43.860854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:23:43.862509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:43.934812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a87f9-1fef-4d3b-bc60-28ad1ab81e05 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:23:43.941088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:23:43.941731 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:43.942016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:43.942289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:43.951053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:23:43.952696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:23:43.961801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1221/2434] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:23:52.606667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:23:52.608203 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:52.634329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76630740-3344-4d8f-aff9-87b76dda7ba2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:23:52.640031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:23:52.640987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:52.641331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:52.641809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] PATH_INFO: `/v3/users/2119337dc0cf4c5c9a11cf7e762fcd3c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:52.643265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2119337dc0cf4c5c9a11cf7e762fcd3c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:23:52.645555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:23:52.700044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2119337dc0cf4c5c9a11cf7e762fcd3c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:52.700668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2119337dc0cf4c5c9a11cf7e762fcd3c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:52.708255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2119337dc0cf4c5c9a11cf7e762fcd3c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:52.708922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2119337dc0cf4c5c9a11cf7e762fcd3c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:52.722769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2119337dc0cf4c5c9a11cf7e762fcd3c'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:52.739364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1214/2435] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:23:52 2022] DELETE /identity/v3/users/2119337dc0cf4c5c9a11cf7e762fcd3c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:23:52.953492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:23:52.970799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cc02e-9274-4157-95ea-4c7a5ec9e81d service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:23:53.061761 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:23:53.063403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:53.064349 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:53.064927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] PATH_INFO: `/v3/projects/ddf6034e4fe740b19807be75cb04bce5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:53.066310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf6034e4fe740b19807be75cb04bce5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:23:53.073578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:23:53.132399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf6034e4fe740b19807be75cb04bce5', 'request_id': 'req-78b93220-105a-40be-a74c-3b106a3b749c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.140042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf6034e4fe740b19807be75cb04bce5', 'request_id': 'req-78b93220-105a-40be-a74c-3b106a3b749c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.150195 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf6034e4fe740b19807be75cb04bce5', 'request_id': 'req-78b93220-105a-40be-a74c-3b106a3b749c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.166867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] The token cache is being invalidate because project ddf6034e4fe740b19807be75cb04bce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:23:53.167173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.167830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.169788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1222/2436] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:23:52 2022] DELETE /identity/v3/projects/ddf6034e4fe740b19807be75cb04bce5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:23:53.174207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:23:53.175688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:53.189612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac040d1-dac0-4ec3-a057-55fa44da7bbe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:23:53.249273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:23:53.249856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:53.250136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:53.250405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] PATH_INFO: `/v3/users/573fc0ec10864754bab9f8a9a60693a5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:53.251780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573fc0ec10864754bab9f8a9a60693a5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:23:53.253699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:23:53.276234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:23:53.307377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573fc0ec10864754bab9f8a9a60693a5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.308125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573fc0ec10864754bab9f8a9a60693a5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.315389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573fc0ec10864754bab9f8a9a60693a5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.316070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573fc0ec10864754bab9f8a9a60693a5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.327837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-63d2f4aa-91fe-442d-a663-c460c39b3ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573fc0ec10864754bab9f8a9a60693a5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.345717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1215/2437] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:23:53 2022] DELETE /identity/v3/users/573fc0ec10864754bab9f8a9a60693a5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:23:53.538222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:23:53.540175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:53.555320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b93220-105a-40be-a74c-3b106a3b749c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:23:53.573988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:23:53.619654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:23:53.620867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:53.621154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:53.621470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] PATH_INFO: `/v3/projects/0b421c3d941742e7903fa2e8914d283b` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:53.622776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b421c3d941742e7903fa2e8914d283b)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:23:53.624795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:23:53.672614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b421c3d941742e7903fa2e8914d283b', 'request_id': 'req-5c53fce7-509e-4eee-a851-b97422da4ba1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.678717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b421c3d941742e7903fa2e8914d283b', 'request_id': 'req-5c53fce7-509e-4eee-a851-b97422da4ba1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.685106 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b421c3d941742e7903fa2e8914d283b', 'request_id': 'req-5c53fce7-509e-4eee-a851-b97422da4ba1'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.698836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] The token cache is being invalidate because project 0b421c3d941742e7903fa2e8914d283b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:23:53.698836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.699485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:23:53.701431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1223/2438] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:23:53 2022] DELETE /identity/v3/projects/0b421c3d941742e7903fa2e8914d283b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:23:53.783368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:23:53.784158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:23:53.784323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:23:53.806056 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] MFA Rules not processed for user `513cbdb59f2b4466963279513c7f8b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:23:53.807444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:23:53.868435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1216/2439] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:23:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:03.527293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:03.528829 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:03.585853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fce7-509e-4eee-a851-b97422da4ba1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:03.589411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:03.590054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:03.590335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:03.590607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:03.592398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:03.656764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:03.671947 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:03.674255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c697f4-bab9-4355-921c-133b24845611 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:03.685557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1224/2440] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:03.769013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:03.778372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38364c1-24ad-4998-b14a-44cf7eecc764 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:03.782014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:03.782666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:03.782946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:03.783216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:03.785034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:03.799234 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:03.800710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:03.812985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1217/2441] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:03.826267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:03.826562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:03.826845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:03.845982 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] MFA Rules not processed for user `7708b4816bcf4301ba7cc826d533aefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:03.847200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:03.906519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1225/2442] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:13.806849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=8,keystone:SELECT=51,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:13.898797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=29 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:20.347170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:20.348665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:20.362419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03f621-a9e3-4d55-9288-572973cd83c9 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:20.367910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:20.368553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:20.368832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:20.369104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:20.371051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:20.428445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:20.430026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:20.438999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1218/2443] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:30.434702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=28 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:33.002854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:33.004754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:33.082362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18554e3-bf46-4951-825c-6087cb561562 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:33.087494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:33.088370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:33.088656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:33.088933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] PATH_INFO: `/v3/users/1009f79ba74e44a1be8781d74e95baa6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:33.090118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1009f79ba74e44a1be8781d74e95baa6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:33.092138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:33.137938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1009f79ba74e44a1be8781d74e95baa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.145276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1009f79ba74e44a1be8781d74e95baa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.145935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1009f79ba74e44a1be8781d74e95baa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.153407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1009f79ba74e44a1be8781d74e95baa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.154022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7409a7fd-4d0a-44c3-a214-80bad669be1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1009f79ba74e44a1be8781d74e95baa6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.171654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1226/2444] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:24:33 2022] DELETE /identity/v3/users/1009f79ba74e44a1be8781d74e95baa6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:24:33.345829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:33.356433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d63dee-b8ab-46ed-899c-1fcbaf6ee38b service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:33.419747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:33.420617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:33.420922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:33.421193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] PATH_INFO: `/v3/projects/b14b528255fe4423b6129876f7cf471e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:33.422347 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14b528255fe4423b6129876f7cf471e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:33.424334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:33.468151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14b528255fe4423b6129876f7cf471e', 'request_id': 'req-9a65d216-2195-49e2-8a56-f7edda21585f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.474745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14b528255fe4423b6129876f7cf471e', 'request_id': 'req-9a65d216-2195-49e2-8a56-f7edda21585f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.481874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14b528255fe4423b6129876f7cf471e', 'request_id': 'req-9a65d216-2195-49e2-8a56-f7edda21585f'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.491908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] The token cache is being invalidate because project b14b528255fe4423b6129876f7cf471e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:24:33.492189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.492827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:24:33.494574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1219/2445] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:24:33 2022] DELETE /identity/v3/projects/b14b528255fe4423b6129876f7cf471e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:24:37.851524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541360-73c4-4b71-94f2-44110893acba None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:37.851981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541360-73c4-4b71-94f2-44110893acba None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:37.852078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541360-73c4-4b71-94f2-44110893acba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:37.869821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1e541360-73c4-4b71-94f2-44110893acba None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:37.871013 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e541360-73c4-4b71-94f2-44110893acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:37.925484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1227/2446] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:24:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:37.937252 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:37.939168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:38.010413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65d216-2195-49e2-8a56-f7edda21585f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:38.015622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cb356-6943-4051-8d27-19986301b58d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:38.016263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb356-6943-4051-8d27-19986301b58d service swift] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:38.016540 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb356-6943-4051-8d27-19986301b58d service swift] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:38.017527 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cb356-6943-4051-8d27-19986301b58d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:38.025602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb356-6943-4051-8d27-19986301b58d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:38.027182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cb356-6943-4051-8d27-19986301b58d service swift] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:38.035029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1220/2447] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:24:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:24:47.195396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541360-73c4-4b71-94f2-44110893acba None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:47.196935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e541360-73c4-4b71-94f2-44110893acba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:47.229233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:47.262012 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541360-73c4-4b71-94f2-44110893acba None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:47.265784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:47.267006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:47.267326 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:47.267607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:47.269423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:47.279336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:47.280786 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:47.289344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1228/2448] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:48.031859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:51.538612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cb356-6943-4051-8d27-19986301b58d service swift] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:51.541410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-066cb356-6943-4051-8d27-19986301b58d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:51.629628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cb356-6943-4051-8d27-19986301b58d service swift] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:51.635899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:51.636547 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:51.636828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:51.637102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:51.639578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:51.653812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:51.655589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4dfae2-8731-4346-b5d3-33c23266929c service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:51.667877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1221/2449] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:51.763367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:51.765021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:51.775813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ded3b6-459e-4812-9532-092fd1403652 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:51.779368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:51.779999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:51.780281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:51.780555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:51.782286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:51.791354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:51.792776 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebbf20-d1b9-407c-9f1b-115239fe3f5a service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:51.801297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1229/2450] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:51.899297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:51.899591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:51.899874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:51.901297 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1222/2451] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:24:51.906975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ef64-df7b-4fde-8b48-f9b1bb235b51 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:51.907980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ef64-df7b-4fde-8b48-f9b1bb235b51 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:51.908264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201ef64-df7b-4fde-8b48-f9b1bb235b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:51.926382 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0201ef64-df7b-4fde-8b48-f9b1bb235b51 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:51.927617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0201ef64-df7b-4fde-8b48-f9b1bb235b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:51.936729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1230/2452] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:24:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:51.948889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:51.950704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:52.009572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5db4f-00aa-4276-87ef-8d8130a91145 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:52.013073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:52.013714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.013993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.014511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.025446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:52.027037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a23988-6ee0-431c-9256-fc038a79a6a3 service glance] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:52.038662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1223/2453] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:52.106432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44932-2a40-4da7-826e-8135b231238f None None] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.106740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44932-2a40-4da7-826e-8135b231238f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.107049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44932-2a40-4da7-826e-8135b231238f None None] PATH_INFO: `/` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.108442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1231/2454] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:24:52.113474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276fe4-9bd3-46e0-b2f5-af6e7c8f1040 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.113827 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276fe4-9bd3-46e0-b2f5-af6e7c8f1040 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.114048 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276fe4-9bd3-46e0-b2f5-af6e7c8f1040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.132492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-13276fe4-9bd3-46e0-b2f5-af6e7c8f1040 None None] MFA Rules not processed for user `15a31aebf6b8454cb69d574dbb0c6409`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:52.133693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13276fe4-9bd3-46e0-b2f5-af6e7c8f1040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:52.140379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1224/2455] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:52.152787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44932-2a40-4da7-826e-8135b231238f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:52.164952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44932-2a40-4da7-826e-8135b231238f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:52.168352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:52.168976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.169255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.169525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.176940 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:52.178357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92974890-d434-4e94-9be9-5e2c26a54ecb service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:52.185915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1232/2456] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:52.272027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708394d-5598-4398-834d-fb847480bb5b None None] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.272316 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708394d-5598-4398-834d-fb847480bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.272595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708394d-5598-4398-834d-fb847480bb5b None None] PATH_INFO: `/` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.274006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1225/2457] 10.222.0.39 () {54 vars in 921 bytes} [Thu May 26 03:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 03:24:52.279171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.279493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.279792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.298002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] MFA Rules not processed for user `2e9e7e21bd294ab389a68c50b4c2131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:52.299217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:52.308204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1233/2458] 10.222.0.39 () {58 vars in 1018 bytes} [Thu May 26 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:52.318865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708394d-5598-4398-834d-fb847480bb5b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:52.330145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708394d-5598-4398-834d-fb847480bb5b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:52.333882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:52.334511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.334790 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.335061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.345251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:52.346678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:52.357123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1226/2459] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:52.418149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:52.419599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:52.488020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc956-8ae9-4f0b-9a60-35d2acb46547 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:52.493082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:52.493770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:52.494049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:52.494327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:52.496150 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:52.508926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:52.510516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6fd7c-ae66-4db1-8180-c5487629451b service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:52.521470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1234/2460] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:55.295867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:55.310163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d3f67-afc0-4e0e-b056-945900841605 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:55.316230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:55.316861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:55.317136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:55.317404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:55.319147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:55.385475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:55.387256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:55.397211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1227/2461] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:55.949384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:55.950507 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:55.951026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:55.972561 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:24:55.973789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:55.995441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1235/2462] 10.222.0.39 () {58 vars in 1009 bytes} [Thu May 26 03:24:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:24:55.997002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:55.998173 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:24:55.998466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:56.014585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594df16-fbf3-44c3-b47b-0e0952669270 service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:56.021057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:56.022054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:56.022504 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:56.022836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782853c2-6a1e-4c7c-84b6-4b6e0781e86d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:24:56.023121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:56.025209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:24:56.033565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:24:56.035400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:24:56.036050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:24:56.036491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:24:56.037991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:24:56.044142 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:56.046144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277abdb5-cd4f-4388-bc92-952fc3137237 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:56.058636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:24:56.061861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:24:56.066083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1228/2463] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:24:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:24:56.090242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1236/2464] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:25:06.054595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:06.075204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=74 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:13.514123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:13.514583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:13.514699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:13.536319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] MFA Rules not processed for user `41c0ab6341464ac388e96852bb400679`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:13.537603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:13.545181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1229/2465] 10.222.0.39 () {58 vars in 967 bytes} [Thu May 26 03:25:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:13.558645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:13.569533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58760095-2df0-4e3f-a252-6189c5f26e12 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:13.574556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6eb06c62ee340a0b70fbb88e9f3d09b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:13.575189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:13.575501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:13.575770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:13.585641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:13.587520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:13.596671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1237/2466] 10.222.0.39 () {60 vars in 1384 bytes} [Thu May 26 03:25:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 03:25:18.691705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:18.701027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ca69f-45d9-463a-8360-1c91ee294521 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:18.704525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:18.705756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:18.706035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:18.706448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:18.708242 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:18.769447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:18.770886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:18.780217 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1230/2467] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:21.886309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:21.888025 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:21.904290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f526d0-5b93-489c-b44d-171a618a1fc8 service swift] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:21.909481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:21.910345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:21.910632 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:21.910909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] PATH_INFO: `/v3/users/7708b4816bcf4301ba7cc826d533aefc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:21.912140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7708b4816bcf4301ba7cc826d533aefc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:21.914067 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:21.960509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7708b4816bcf4301ba7cc826d533aefc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:21.967209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7708b4816bcf4301ba7cc826d533aefc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:21.967897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7708b4816bcf4301ba7cc826d533aefc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:21.975913 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7708b4816bcf4301ba7cc826d533aefc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:21.976531 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-56d629ca-57f8-4a29-9c83-7fd65d515ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7708b4816bcf4301ba7cc826d533aefc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:21.994247 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1238/2468] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:25:21 2022] DELETE /identity/v3/users/7708b4816bcf4301ba7cc826d533aefc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:22.167502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.178998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132beb-a5f7-4ba8-9725-1f632eed63ac service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.235380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.236259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.237216 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.237499 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] PATH_INFO: `/v3/projects/2d9bfd3a2b544d74bb581b5bd13e8ecc` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.238835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9bfd3a2b544d74bb581b5bd13e8ecc)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.241427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.289164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9bfd3a2b544d74bb581b5bd13e8ecc', 'request_id': 'req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:22.295556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9bfd3a2b544d74bb581b5bd13e8ecc', 'request_id': 'req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:22.303603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9bfd3a2b544d74bb581b5bd13e8ecc', 'request_id': 'req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:22.313817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] The token cache is being invalidate because project 2d9bfd3a2b544d74bb581b5bd13e8ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:25:22.314244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:22.315065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-169d747a-4e4b-4e63-b1c1-97d6a5a05e3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:22.316629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1231/2469] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:25:22 2022] DELETE /identity/v3/projects/2d9bfd3a2b544d74bb581b5bd13e8ecc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:22.321546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.321908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.322222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.343528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:22.345383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:22.406330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1239/2470] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:22.414650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.414939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.415215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.433899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:22.435099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:22.442954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1232/2471] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:22.447126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.456892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166d2ba-8aba-44e1-a741-9d01347fd1a4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.461446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.462094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.462378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.462654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.463452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:25:22.463762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.465367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.477451 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1240/2472] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:25:22.481657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.491513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0980-5520-4fde-b1e6-b6814ce8bac8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.496126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.496761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.497041 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.497314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.497994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.499937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.508042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:22.528076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1233/2473] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:22 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:22.532209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.542894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcadfe0-ac8c-4d5d-9499-3a0c61ed0237 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.586400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:22.602158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.602757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.603036 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.603370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.604113 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.606014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.624748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1241/2474] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:22 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:22.629322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.639786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acee876-3b3e-415b-bbff-c3de7abbd380 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.644934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.645586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.645870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.646155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.646868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.648505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.658873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1234/2475] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:25:22.662830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.672638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd19ce4-4f3b-48da-baf7-373b5063f765 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.677253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.677893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.678172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.678446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.679183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.680821 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d2f34-988a-424b-997b-55d81d4f0b6e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.690504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1242/2476] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:22.694455 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.704475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc548a-5685-48ff-a0b8-ccdfd120970a admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.710045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.710925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.711199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.711500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] PATH_INFO: `/v3/projects/3676f20a4e0d4ec8be71d5aaab607aa7/users/9e41cedaabc54e408cdddee2e5cb140d/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.727951 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3676f20a4e0d4ec8be71d5aaab607aa7, user_id=9e41cedaabc54e408cdddee2e5cb140d, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.730667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.745409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1235/2477] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:22 2022] PUT /identity/v3/projects/3676f20a4e0d4ec8be71d5aaab607aa7/users/9e41cedaabc54e408cdddee2e5cb140d/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:22.750521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364d85-8923-48bf-9a5f-da5664924d1b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.750816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364d85-8923-48bf-9a5f-da5664924d1b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.751096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364d85-8923-48bf-9a5f-da5664924d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.770859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-e8364d85-8923-48bf-9a5f-da5664924d1b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:22.772086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e8364d85-8923-48bf-9a5f-da5664924d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:22.831085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1243/2478] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:22.840634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:22.842326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:22.898946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dca73c-ad60-4324-afc8-885943050375 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:22.902530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:22.903163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:22.903480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:22.903764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:22.912568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:22.914029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:22.923510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1236/2479] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:27.154509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2f5e-c1d4-4a16-b014-bfbc5b88a8c4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:27.155010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2f5e-c1d4-4a16-b014-bfbc5b88a8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:27.155111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2f5e-c1d4-4a16-b014-bfbc5b88a8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:27.186108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-df8c2f5e-c1d4-4a16-b014-bfbc5b88a8c4 None None] MFA Rules not processed for user `9e41cedaabc54e408cdddee2e5cb140d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:27.187572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-df8c2f5e-c1d4-4a16-b014-bfbc5b88a8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:27.279853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1244/2480] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:27.294028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:27.296105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:27.382254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c81cdc-b19e-4d01-b6b1-4f38400ad84b service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:27.388623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:27.389622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:27.390004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:27.390348 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:27.400402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:27.402512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:27.412778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1237/2481] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:28.868047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:28.868600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:28.868600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:28.887781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] MFA Rules not processed for user `9e41cedaabc54e408cdddee2e5cb140d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:28.889240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:28.897020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1245/2482] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:36.270078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:36.279469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bc968-c6d6-4fae-8ded-e990ae54effb service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:36.283068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:36.283738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:36.284014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:36.284285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:36.286031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:36.297544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:36.299486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:36.308869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1238/2483] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:38.881030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:39.483156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:39.484898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:39.572529 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:39.574022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:39.587465 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5276701-faf3-4edb-a888-a1502ee057de service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:39.592342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:39.593478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:39.593760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:39.594034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] PATH_INFO: `/v3/users/513cbdb59f2b4466963279513c7f8b06` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:39.596125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=513cbdb59f2b4466963279513c7f8b06)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:39.598780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:39.602661 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1da1b-22d0-484b-94df-85e2a662495e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:39.609236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:39.609900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:39.610371 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:39.611288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:39.613329 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:39.633141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:39.634856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:39.635211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:39.648194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1246/2484] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:39.667118 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513cbdb59f2b4466963279513c7f8b06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:39.667824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513cbdb59f2b4466963279513c7f8b06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:39.677875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513cbdb59f2b4466963279513c7f8b06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:39.678917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513cbdb59f2b4466963279513c7f8b06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:39.692639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d46816a4-bca7-4b14-9104-119f8b5481ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513cbdb59f2b4466963279513c7f8b06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:39.704899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1239/2485] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:25:39 2022] DELETE /identity/v3/users/513cbdb59f2b4466963279513c7f8b06 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:39.882595 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:39.894444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c84a6-6e41-4c92-ab66-6a336ba997c2 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:39.952671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:39.953544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:39.953824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:39.954124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] PATH_INFO: `/v3/projects/921ab4465c1b48758a03fe0411940778` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:39.955981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=921ab4465c1b48758a03fe0411940778)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:39.957897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.003922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '921ab4465c1b48758a03fe0411940778', 'request_id': 'req-6e18e155-2eea-4076-a667-5d2ba9989601'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:40.009520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '921ab4465c1b48758a03fe0411940778', 'request_id': 'req-6e18e155-2eea-4076-a667-5d2ba9989601'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:40.016274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '921ab4465c1b48758a03fe0411940778', 'request_id': 'req-6e18e155-2eea-4076-a667-5d2ba9989601'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:40.033558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] The token cache is being invalidate because project 921ab4465c1b48758a03fe0411940778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:25:40.033846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:40.034479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6e18e155-2eea-4076-a667-5d2ba9989601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:40.036085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1247/2486] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:25:39 2022] DELETE /identity/v3/projects/921ab4465c1b48758a03fe0411940778 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:40.041129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.041490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.041778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.060771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:40.061977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:40.121146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1240/2487] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:40.132206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.132493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.132771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.152803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:40.154095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:40.162655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1248/2488] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:40.166989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.177194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec0c8f-f099-4d35-be47-d3718f417917 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.181798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.182452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.182797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.183003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.183761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:25:40.184063 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.185659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.197908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1241/2489] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:25:40.202488 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.212733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9743b6-0cdf-43d9-87d1-375749c4ce80 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.217440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.218105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.218390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.218668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.219403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.221551 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.232311 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:40.261646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1249/2490] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:40.266361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.277254 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf83025-f9b2-4793-83f0-4644e39606c7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.335747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.336434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.336723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.337002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.337735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.339847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.362042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1242/2491] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:40.366367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.377514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb69f4-508a-4b25-bb94-a4d20ece6d45 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.382123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.382799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.383087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.383407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.384134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.385770 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.396685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1250/2492] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:40.400699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.411835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a802c19-de0a-4826-94a8-514e340795aa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.416665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.417341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.417623 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.417998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.418906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.420656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e575b3a-1149-4600-962a-a909218135e8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.432349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1243/2493] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:40.437903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.448262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa3cae-b408-48a3-9440-5eb8ca477762 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.452836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.453707 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.453988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.454397 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] PATH_INFO: `/v3/projects/7b9cd10c9bab483281f678ca42c531e8/users/0882a7dadf774412b7d1fafa931cabfa/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.469271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9cd10c9bab483281f678ca42c531e8, user_id=0882a7dadf774412b7d1fafa931cabfa, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.472477 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.486872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1251/2494] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:40 2022] PUT /identity/v3/projects/7b9cd10c9bab483281f678ca42c531e8/users/0882a7dadf774412b7d1fafa931cabfa/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:40.497896 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.498183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.498458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.511469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:40.530397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] MFA Rules not processed for user `0882a7dadf774412b7d1fafa931cabfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:40.531707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:40.600240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1244/2495] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:40.604744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.670008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183069-dd77-4b6e-a0cb-1aa3f3011d3d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.675415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.676074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.676360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.676645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.677352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.679294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.709663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1252/2496] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:40.711104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.722571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294836ff-c1f4-4eab-bb9f-dc6593a044a1 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.779859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.780525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.780811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.781086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.781931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.784462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.807925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1245/2497] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:40 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:40.809064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.819749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59f4d8-742f-41be-87cd-85c9ea627758 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.824406 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.825064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.825344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.825618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.826337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.828099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.845264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1253/2498] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:25:40.846723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.857656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003082c-c646-4f73-b297-80038e40edab admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.863014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.863919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.864201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.864477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] PATH_INFO: `/v3/projects/d04355b46b74472da818e0c2ffbc7b1c/users/5e6c2b067be941fbb26d7e0e7cf0f48c/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.880036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04355b46b74472da818e0c2ffbc7b1c, user_id=5e6c2b067be941fbb26d7e0e7cf0f48c, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.882856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.897286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1246/2499] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:40 2022] PUT /identity/v3/projects/d04355b46b74472da818e0c2ffbc7b1c/users/5e6c2b067be941fbb26d7e0e7cf0f48c/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:40.901314 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:40.912219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9962fed-ed0e-45b0-9140-9c5c9d6fe3f7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:40.972906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:40.973478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:40.973759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:40.974082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:40.974804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:40.976418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e102e-7472-4547-8c61-04b4f8a9917d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:40.986441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1254/2500] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:25:40.990737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:41.001787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9bdf9-a8b7-47f1-8f2c-1c38a1b21932 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:41.006679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:41.007577 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:41.007856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:41.008154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] PATH_INFO: `/v3/projects/d04355b46b74472da818e0c2ffbc7b1c/users/5e6c2b067be941fbb26d7e0e7cf0f48c/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:41.010127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d04355b46b74472da818e0c2ffbc7b1c, user_id=5e6c2b067be941fbb26d7e0e7cf0f48c, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:41.012806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:41.024555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1247/2501] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:40 2022] PUT /identity/v3/projects/d04355b46b74472da818e0c2ffbc7b1c/users/5e6c2b067be941fbb26d7e0e7cf0f48c/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:41.035663 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:41.035954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:41.036235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:41.066872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] MFA Rules not processed for user `5e6c2b067be941fbb26d7e0e7cf0f48c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:41.068271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:41.077164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:41.150276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1255/2502] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:41 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:41.162968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:41.164765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:41.222391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:41.227445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4864d-72a2-4ea0-b8b8-9ee343bc442c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:41.232436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:41.233077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:41.233359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:41.234012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:41.244198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:41.246310 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:41.257218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1248/2503] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:41 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:41.330149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:41.340920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b61cbe-e562-4ba8-b310-cd90e5fb5909 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:41.344846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:41.346065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:41.346348 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:41.346620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:41.412584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:41.414023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:41.426283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1256/2504] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:41 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:51.253279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:51.418916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=54 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:53.662601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:53.664035 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:53.722238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb3ba7-7dec-4516-b37f-72d72ce4ca19 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:53.725742 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:53.726374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:53.726649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:53.726916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:53.728735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:53.791042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:53.792505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378377e0-9027-4501-8706-93a03e35f277 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:53.801993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1249/2505] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:56.312290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:56.325635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668caa45-647f-4081-aacf-8ff8469dee2f service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:56.333672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:56.334751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:56.335034 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:56.335340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] PATH_INFO: `/v3/users/0882a7dadf774412b7d1fafa931cabfa` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:56.336662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0882a7dadf774412b7d1fafa931cabfa)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:56.338737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:56.407599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0882a7dadf774412b7d1fafa931cabfa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.416679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0882a7dadf774412b7d1fafa931cabfa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.417527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0882a7dadf774412b7d1fafa931cabfa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.425524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0882a7dadf774412b7d1fafa931cabfa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.426171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0882a7dadf774412b7d1fafa931cabfa'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.445575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1257/2506] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:25:56 2022] DELETE /identity/v3/users/0882a7dadf774412b7d1fafa931cabfa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:56.449952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:56.450413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:56.450507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:56.470408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:56.472101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:56.534624 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1250/2507] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:56.544910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:56.546313 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:56.614042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668163e-4e05-4dbe-b698-9131ddbe29e7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:56.623562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:56.624492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:56.625952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:56.626538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:56.644463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:56.646692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:56.658487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1258/2508] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:25:56.781832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:56.783323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:56.796336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b1a1b-a5c3-4510-a066-65aea7e613de None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:56.801007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:56.801871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:56.802149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:56.802417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] PATH_INFO: `/v3/users/3cf837e2ab5b4e58844a1023cf5f9237` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:56.803634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf837e2ab5b4e58844a1023cf5f9237)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:56.805720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:56.868830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:56.870272 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf837e2ab5b4e58844a1023cf5f9237'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.871070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf837e2ab5b4e58844a1023cf5f9237'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.880633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf837e2ab5b4e58844a1023cf5f9237'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.881321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf837e2ab5b4e58844a1023cf5f9237'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.900060 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf837e2ab5b4e58844a1023cf5f9237'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:56.920437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1251/2509] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:25:56 2022] DELETE /identity/v3/users/3cf837e2ab5b4e58844a1023cf5f9237 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:56.989643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:56.991126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.008143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6262532-eda7-44d3-8001-d3ef678fc818 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.074396 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.075294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.075578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.075909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] PATH_INFO: `/v3/projects/7b9cd10c9bab483281f678ca42c531e8` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.077100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9cd10c9bab483281f678ca42c531e8)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.079210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.102124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:57.132749 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.134993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.141630 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9cd10c9bab483281f678ca42c531e8', 'request_id': 'req-fdd75d4a-0449-40db-bafb-b8a362b75c67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.148816 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9cd10c9bab483281f678ca42c531e8', 'request_id': 'req-fdd75d4a-0449-40db-bafb-b8a362b75c67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.155931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9cd10c9bab483281f678ca42c531e8', 'request_id': 'req-fdd75d4a-0449-40db-bafb-b8a362b75c67'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.160331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077bb8b-9f97-4788-b483-3dbf62c99a16 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.176228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] The token cache is being invalidate because project 7b9cd10c9bab483281f678ca42c531e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:25:57.176528 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.178716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.180400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1259/2510] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:25:56 2022] DELETE /identity/v3/projects/7b9cd10c9bab483281f678ca42c531e8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:57.184428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.185749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.235330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.236418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.236698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.236969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] PATH_INFO: `/v3/projects/ae4cce0a69044ebcba5f2b5352d04cbf` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.238154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4cce0a69044ebcba5f2b5352d04cbf)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.240162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.256678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75d4a-0449-40db-bafb-b8a362b75c67 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.261293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.262174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.262455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.262725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] PATH_INFO: `/v3/users/5e6c2b067be941fbb26d7e0e7cf0f48c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.264178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6c2b067be941fbb26d7e0e7cf0f48c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.266693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.290698 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4cce0a69044ebcba5f2b5352d04cbf', 'request_id': 'req-25964014-ad52-4072-b606-4d4b927944b8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.296992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4cce0a69044ebcba5f2b5352d04cbf', 'request_id': 'req-25964014-ad52-4072-b606-4d4b927944b8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.305530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4cce0a69044ebcba5f2b5352d04cbf', 'request_id': 'req-25964014-ad52-4072-b606-4d4b927944b8'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.316170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] The token cache is being invalidate because project ae4cce0a69044ebcba5f2b5352d04cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:25:57.316362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6c2b067be941fbb26d7e0e7cf0f48c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.316826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.317064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-25964014-ad52-4072-b606-4d4b927944b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.318703 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1252/2511] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:25:57 2022] DELETE /identity/v3/projects/ae4cce0a69044ebcba5f2b5352d04cbf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:57.323686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.323970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.324250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.327046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6c2b067be941fbb26d7e0e7cf0f48c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.327777 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6c2b067be941fbb26d7e0e7cf0f48c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.335974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6c2b067be941fbb26d7e0e7cf0f48c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.336692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c9db9ca7-f0f9-4ef7-8401-94d15cf43205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6c2b067be941fbb26d7e0e7cf0f48c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.343791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:57.345413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.354315 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1260/2512] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:25:57 2022] DELETE /identity/v3/users/5e6c2b067be941fbb26d7e0e7cf0f48c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:57.395959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:57.411822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1253/2513] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:57.419991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.420355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.420692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.441729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:57.442964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.464092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:57.517926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1261/2514] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:57.519541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.529766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22925d-c4d4-4f38-89a0-b9dd4e62f8b2 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.535762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.536787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.537070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.537341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.538059 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:25:57.538356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.539994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.553232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1254/2515] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:25:57.557651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.567814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672506-d983-4b8e-8cbe-bf11490d9499 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.572565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.573226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.573509 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.573787 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.574487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.576475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.606748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1262/2516] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:57.607935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.619554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7649-ba31-41bb-8e7c-ff0d98e3d032 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.641198 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.643638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.689472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.690153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.690438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.690713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.691589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.693528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.717193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1255/2517] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:57.722080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.725016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800211-68b6-4bbe-b289-9f7c0b3c8d77 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.729781 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.730699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.731010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.731422 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] PATH_INFO: `/v3/projects/d04355b46b74472da818e0c2ffbc7b1c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.731644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ff031-a52b-42b4-80bb-85a295bb8998 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.732638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d04355b46b74472da818e0c2ffbc7b1c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.734756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.737346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.737999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.738280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.738554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.739302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.740998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.755825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1256/2518] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:25:57.756890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.767476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091fa3-7456-4131-9070-b2af6fc542a0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.772817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.773456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.773735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.774008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.774714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.776334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.788152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1257/2519] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:57.791967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd04355b46b74472da818e0c2ffbc7b1c', 'request_id': 'req-b57e0928-f570-4a74-8178-22ad2b1f7753'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.792103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.798091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd04355b46b74472da818e0c2ffbc7b1c', 'request_id': 'req-b57e0928-f570-4a74-8178-22ad2b1f7753'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.801688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6ed84-fe84-43dd-829a-5f69f0c1fa57 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:57.804569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd04355b46b74472da818e0c2ffbc7b1c', 'request_id': 'req-b57e0928-f570-4a74-8178-22ad2b1f7753'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.819380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] The token cache is being invalidate because project d04355b46b74472da818e0c2ffbc7b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:25:57.819744 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.820327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b57e0928-f570-4a74-8178-22ad2b1f7753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:25:57.821899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1263/2520] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:25:57 2022] DELETE /identity/v3/projects/d04355b46b74472da818e0c2ffbc7b1c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:57.826656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cca03d-b968-494b-b648-33580ea4032e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.826942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cca03d-b968-494b-b648-33580ea4032e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.827217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cca03d-b968-494b-b648-33580ea4032e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.846423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-59cca03d-b968-494b-b648-33580ea4032e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:57.847657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-59cca03d-b968-494b-b648-33580ea4032e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.859545 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:57.860440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.860716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.860989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] PATH_INFO: `/v3/projects/680109befda549beae35fd3c152185a4/users/678e5c9c8cd2411f908bfa0e05f3827c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.874801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=680109befda549beae35fd3c152185a4, user_id=678e5c9c8cd2411f908bfa0e05f3827c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:57.877621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a55164-f550-4610-8fbf-b3dc0c3f6445 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:57.878840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:57.891913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1258/2521] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:57 2022] PUT /identity/v3/projects/680109befda549beae35fd3c152185a4/users/678e5c9c8cd2411f908bfa0e05f3827c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:57.896763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.897042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.897317 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.914398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:57.915722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.917730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1264/2522] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:57.925549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:57.925840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:57.926114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:57.941101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:57.944257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:57.945469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:57.975012 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1259/2523] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:57.984493 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:57.986380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:58.007303 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1265/2524] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:58.011433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.021659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812fa11-4651-439d-9987-efb2860fa876 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.026407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.027068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.027389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.027671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.028392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:25:58.028729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.030614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.045833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1266/2525] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:25:58.046760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.050353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4147-7ba1-46d4-856a-a267fc6ad9b5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.053989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.054632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.054909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.055180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.056230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4b196-76fc-4192-a5e9-0322440dbf76 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.060812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.061435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.061717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.061997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.062665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.064569 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.065617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.065997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.075634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1260/2526] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:25:58.097078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1267/2527] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:58.098246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.108264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264f469-3fc5-4dab-a229-38c6eb8b98f8 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.166748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.167439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.167718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.167990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.168668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.170576 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.190811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1261/2528] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:58.194922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.205837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075199-6b79-444f-ac2a-52feba5b40f0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.210750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.211460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.211740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.212003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.212732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.214334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.225482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1268/2529] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:58.229391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.239677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacd481-aa53-4bb0-8cb3-405c169cfe07 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.244798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.245570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.246515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.246515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.247008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.248778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d3078-d635-477f-9920-be0591edeec2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.264904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1262/2530] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:25:58.266598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.277139 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7afba-9b08-4571-a1e7-0e9fc57a1394 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.282322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.283208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.283520 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.283801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] PATH_INFO: `/v3/projects/104ac255c514430392132200137b579c/users/c92e4fe6179b4ba184ffdacf45736f19/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.299089 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=104ac255c514430392132200137b579c, user_id=c92e4fe6179b4ba184ffdacf45736f19, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.301931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.315710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1269/2531] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:58 2022] PUT /identity/v3/projects/104ac255c514430392132200137b579c/users/c92e4fe6179b4ba184ffdacf45736f19/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:58.326738 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.327030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.327338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.354962 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] MFA Rules not processed for user `c92e4fe6179b4ba184ffdacf45736f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:58.356242 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:58.419036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:58.450901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1263/2532] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:25:58.455284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.524440 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b07-e6cb-4ba3-8845-ef3c0c071ce1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.529164 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.529833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.530123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.530400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.531100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.533237 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.541469 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:58.565354 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1270/2533] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:58.566537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.577555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942a56c-238e-4e97-93db-1d3e213246d4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.636701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.637397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.637682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.637956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.638903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.641531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.667033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1264/2534] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:58.668362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.679412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d770c-00f0-4ebb-8322-5e6bd81a8502 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.684901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.685607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.685892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.686167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.686920 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.688659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ac0f8-4ca9-4240-845a-bd6a7366cf92 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.700209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1271/2535] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:58.705079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.724076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96582683-6e3d-44ac-8d49-80daa8bf57c1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.729518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.730599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.731030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.731491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] PATH_INFO: `/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2/users/0fc060be899c467fab7abe95d09dfcb5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.737564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.737849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.738126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.748480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465df59e06ab4b77bdd1970bab0ccbd2, user_id=0fc060be899c467fab7abe95d09dfcb5, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.751415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.766838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1265/2536] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:58 2022] PUT /identity/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2/users/0fc060be899c467fab7abe95d09dfcb5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:58.770645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.775884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] MFA Rules not processed for user `678e5c9c8cd2411f908bfa0e05f3827c`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:58.777086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:58.839020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b45ab-eddb-4f7e-abeb-313dddcb1d00 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.843959 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.844814 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.845095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.845371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.846239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.847888 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.858937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1266/2537] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:58.860853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1272/2538] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:58.864033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.865037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.866459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:58.874274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463354c-4dc5-41d6-ab61-2987e877e1ec admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.880341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.881413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.881856 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.882004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ae823-7db4-4e9b-920b-56eb020732fe None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:58.882434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] PATH_INFO: `/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2/users/0fc060be899c467fab7abe95d09dfcb5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.884722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] RBAC: Authorizing `identity:create_grant(project_id=465df59e06ab4b77bdd1970bab0ccbd2, user_id=0fc060be899c467fab7abe95d09dfcb5, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.886555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:58.887212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.887525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.887804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.888734 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:58.889152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4e957-ad8b-441a-b613-65cabfc610da admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.890691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:58.904691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1267/2539] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:58 2022] PUT /identity/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2/users/0fc060be899c467fab7abe95d09dfcb5/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:58.915379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:58.915682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:58.915957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:58.927279 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1273/2540] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:25:58.930699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:58.949022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] MFA Rules not processed for user `0fc060be899c467fab7abe95d09dfcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:58.950712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:58.971205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:58.997290 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b103f3-fd9c-4994-aad9-c26a184a3a8f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.002367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.003112 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.003497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.003775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.004459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.006413 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.024887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1268/2541] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:25:59.027938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1274/2542] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:25:58 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:25:59.032491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.036492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.038340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:59.043179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c17329-51e8-4b2c-9871-b8912249659e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.048602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.049294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.049699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.049984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.050711 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.052388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.062701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1269/2543] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:59.067620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.077244 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2914b7-a669-4af0-a750-a5fb7881285f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.081367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:59.082195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.083206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.083629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.083925 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] PATH_INFO: `/v3/projects/8eba822af96a45708f06d708efc014b0/users/0014545eb73b4083a4c6d452b4164265/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.100195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eba822af96a45708f06d708efc014b0, user_id=0014545eb73b4083a4c6d452b4164265, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.101929 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63035c7c-cf53-4afd-a9a3-12f78cc6528a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.102894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.105555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.106271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.106559 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.106838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.116586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1270/2544] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:59 2022] PUT /identity/v3/projects/8eba822af96a45708f06d708efc014b0/users/0014545eb73b4083a4c6d452b4164265/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:25:59.119774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.119774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.120740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.128409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1275/2545] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:25:59.187140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e6f15-4b9d-4810-b87d-d8006e00a484 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.192380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.193112 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.193402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.193674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.194397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.196053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.203801 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.211965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1271/2546] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:25:59.212294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d239-880c-4b94-a6a7-9bdc805ae2a8 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.213046 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:25:59.223745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646994f1-3769-4487-b5c9-710b35ab4e46 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:25:59.228386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.229243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.229524 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.229804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] PATH_INFO: `/v3/projects/8eba822af96a45708f06d708efc014b0/users/0014545eb73b4083a4c6d452b4164265/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.231799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eba822af96a45708f06d708efc014b0, user_id=0014545eb73b4083a4c6d452b4164265, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.234441 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875609e-f6b7-4c31-8e0e-2757dfa7611b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.246070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1272/2547] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:25:59 2022] PUT /identity/v3/projects/8eba822af96a45708f06d708efc014b0/users/0014545eb73b4083a4c6d452b4164265/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:25:59.268059 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:25:59.268710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.268986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.269256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.328908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:25:59.330373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:25:59.339973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1276/2548] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:25:59.874791 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:25:59.875358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:25:59.875582 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:25:59.902103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] MFA Rules not processed for user `0014545eb73b4083a4c6d452b4164265`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:25:59.903412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:25:59.960097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:25:59.980723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1273/2549] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:01.289990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:01.300845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead044a3-c32a-4171-95f2-0f92c02058de service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:01.357728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:01.358607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:01.358891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:01.359170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] PATH_INFO: `/v3/users/678e5c9c8cd2411f908bfa0e05f3827c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:01.360510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=678e5c9c8cd2411f908bfa0e05f3827c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:01.362450 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:01.402230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:01.412647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '678e5c9c8cd2411f908bfa0e05f3827c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.419590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '678e5c9c8cd2411f908bfa0e05f3827c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.420243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '678e5c9c8cd2411f908bfa0e05f3827c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.427713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '678e5c9c8cd2411f908bfa0e05f3827c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.429587 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '678e5c9c8cd2411f908bfa0e05f3827c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.448260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1277/2550] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:01 2022] DELETE /identity/v3/users/678e5c9c8cd2411f908bfa0e05f3827c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:01.619898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:01.631915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c696cb8-35a6-48fc-a99d-0acb4ac3b519 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:01.689709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:01.690473 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:01.690754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:01.691030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] PATH_INFO: `/v3/projects/680109befda549beae35fd3c152185a4` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:01.692236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=680109befda549beae35fd3c152185a4)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:01.694156 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:01.742718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680109befda549beae35fd3c152185a4', 'request_id': 'req-f7857f01-41e4-4d82-922c-9fdd29f05aaa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.748770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680109befda549beae35fd3c152185a4', 'request_id': 'req-f7857f01-41e4-4d82-922c-9fdd29f05aaa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.756799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680109befda549beae35fd3c152185a4', 'request_id': 'req-f7857f01-41e4-4d82-922c-9fdd29f05aaa'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.767439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] The token cache is being invalidate because project 680109befda549beae35fd3c152185a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:01.767732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.768378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.769910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1274/2551] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:01 2022] DELETE /identity/v3/projects/680109befda549beae35fd3c152185a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:01.773928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:01.775344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:01.788269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbae08-741b-4c75-af99-6f4a1e2bac42 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:01.844463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:01.845321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:01.845600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:01.845872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] PATH_INFO: `/v3/users/0014545eb73b4083a4c6d452b4164265` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:01.847024 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0014545eb73b4083a4c6d452b4164265)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:01.848980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:01.893945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0014545eb73b4083a4c6d452b4164265'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.900684 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0014545eb73b4083a4c6d452b4164265'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.901361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0014545eb73b4083a4c6d452b4164265'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.909049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0014545eb73b4083a4c6d452b4164265'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.909750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0014545eb73b4083a4c6d452b4164265'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:01.927213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1278/2552] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:01 2022] DELETE /identity/v3/users/0014545eb73b4083a4c6d452b4164265 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:02.081475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.083155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.096672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857f01-41e4-4d82-922c-9fdd29f05aaa admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.108000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.109832 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.148321 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:02.168425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.169298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.169580 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.169853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] PATH_INFO: `/v3/users/c92e4fe6179b4ba184ffdacf45736f19` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.171028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92e4fe6179b4ba184ffdacf45736f19)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.173253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.180524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af67e7-0926-4a76-bcfa-175eb6b0d1a7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.185126 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.185984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.186263 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.186541 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] PATH_INFO: `/v3/projects/8eba822af96a45708f06d708efc014b0` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.188262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eba822af96a45708f06d708efc014b0)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.190178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.195484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:02.224617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92e4fe6179b4ba184ffdacf45736f19'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.225265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92e4fe6179b4ba184ffdacf45736f19'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.232127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92e4fe6179b4ba184ffdacf45736f19'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.233025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92e4fe6179b4ba184ffdacf45736f19'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.239402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eba822af96a45708f06d708efc014b0', 'request_id': 'req-10b29068-3a41-46c4-a6ae-e9ef92212277'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.245703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eba822af96a45708f06d708efc014b0', 'request_id': 'req-10b29068-3a41-46c4-a6ae-e9ef92212277'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.246437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-eb31011c-8cbc-4172-b15a-03e7dddc8319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92e4fe6179b4ba184ffdacf45736f19'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.253591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eba822af96a45708f06d708efc014b0', 'request_id': 'req-10b29068-3a41-46c4-a6ae-e9ef92212277'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.262945 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1275/2553] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:02 2022] DELETE /identity/v3/users/c92e4fe6179b4ba184ffdacf45736f19 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:02.268014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.268354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.268634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.269131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] The token cache is being invalidate because project 8eba822af96a45708f06d708efc014b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:02.269431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.270119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-10b29068-3a41-46c4-a6ae-e9ef92212277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.271891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1279/2554] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:02 2022] DELETE /identity/v3/projects/8eba822af96a45708f06d708efc014b0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:02.277200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4422c-a3e1-4b82-8cc9-96cb6e7539ad None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.277499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4422c-a3e1-4b82-8cc9-96cb6e7539ad None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.277784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4422c-a3e1-4b82-8cc9-96cb6e7539ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.288621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:02.289853 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.298465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-dfd4422c-a3e1-4b82-8cc9-96cb6e7539ad None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:02.299682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-dfd4422c-a3e1-4b82-8cc9-96cb6e7539ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.349628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1276/2555] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:02.359139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.361286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1280/2556] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:02.361425 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.372858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.373449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.374196 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.394989 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:02.396266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.404492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1281/2557] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:02.408478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.417310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf678a-4134-41b6-9311-eea75702ac1d None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.417518 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0cbeb3-6298-41c1-9ae1-c67d9458ff63 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.421346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.422423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.422564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.422977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.423075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.423276 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.423389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.423645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.424368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:02.424680 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.426284 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.432318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.433746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.438340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1282/2558] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:02.442448 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.443864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1277/2559] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:02.452296 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360ec66-373c-4ec9-9986-2441ebcddbce admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.457850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.458502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.458788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.459062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.459782 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.461691 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.493099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1283/2560] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:02.494349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.504874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb181fa-2072-4586-ad58-59b218e56298 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.551956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:02.571446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.572238 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.572517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.572788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.573481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.575432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.601352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1278/2561] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:02.605459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.616907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb60c60-8903-4f7e-9c54-20922ef1cc77 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.622096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.622784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.623065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.623365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.624085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.626038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.640307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1284/2562] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:02.641523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.651498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad773-d577-4db3-bfbb-1ea26e6e48fc admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.656027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.657207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.657486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.657754 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.658505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.660132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.670331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1279/2563] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:02.675086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.684917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35bc02-8aea-4876-ac4d-074717275fc1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.689580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.690452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.690811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.691085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] PATH_INFO: `/v3/projects/5ad2146a47134604b60d2ddbb8e8e1a1/users/6f156fbda63040eb9f75e8b97428fb55/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.694169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:02.709985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad2146a47134604b60d2ddbb8e8e1a1, user_id=6f156fbda63040eb9f75e8b97428fb55, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.713810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d8868-4d92-432e-9748-9d5948af885b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.724676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.726712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.735388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1285/2564] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:02 2022] PUT /identity/v3/projects/5ad2146a47134604b60d2ddbb8e8e1a1/users/6f156fbda63040eb9f75e8b97428fb55/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:02.741118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.741565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.741938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.742587 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84a839-70b1-467c-885a-ad8447129052 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.772103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] MFA Rules not processed for user `6f156fbda63040eb9f75e8b97428fb55`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:02.773848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.804983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.805954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.806231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.806507 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] PATH_INFO: `/v3/projects/104ac255c514430392132200137b579c` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.807843 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=104ac255c514430392132200137b579c)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.810514 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.840447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1286/2565] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:02.845258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.856084 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a3220-9772-41a8-81ff-ad19d7fd40e2 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:02.860801 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104ac255c514430392132200137b579c', 'request_id': 'req-5362e930-f7f3-4b52-8efc-433acbae41d2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.867249 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104ac255c514430392132200137b579c', 'request_id': 'req-5362e930-f7f3-4b52-8efc-433acbae41d2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.875781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104ac255c514430392132200137b579c', 'request_id': 'req-5362e930-f7f3-4b52-8efc-433acbae41d2'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.886371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] The token cache is being invalidate because project 104ac255c514430392132200137b579c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:02.886660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.887333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:02.889230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1280/2566] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:02 2022] DELETE /identity/v3/projects/104ac255c514430392132200137b579c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:02.893581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.894918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.915757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:02.916416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:02.916694 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:02.916966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:02.917649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:02.919567 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:02.945697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1287/2567] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:02.947100 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:02.949774 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:02.951715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:02.962617 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362e930-f7f3-4b52-8efc-433acbae41d2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.018301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639449a-e857-4ebf-9a8d-9c4c1d1beec7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.020996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.021870 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.022153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.022428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] PATH_INFO: `/v3/users/0fc060be899c467fab7abe95d09dfcb5` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.022884 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.023578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.023686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc060be899c467fab7abe95d09dfcb5)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.023867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.024147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.024830 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.025655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.026768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.034442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:03.051478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1288/2568] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:02 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:03.055481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.066136 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58fd21-cce9-4834-9d45-77712b7c823a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.071056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.071784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.072077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.072393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.073217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.074729 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.075387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc060be899c467fab7abe95d09dfcb5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.076128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc060be899c467fab7abe95d09dfcb5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.083685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc060be899c467fab7abe95d09dfcb5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.084432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc060be899c467fab7abe95d09dfcb5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.085602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1289/2569] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:03.089785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.091350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.096635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc060be899c467fab7abe95d09dfcb5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.104269 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee189f6-b3b4-4855-9379-2859b8cf67ec admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.109170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.110622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.110903 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.111178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] PATH_INFO: `/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291/users/6f9142dbec9a4d9dbeba37ac9d85bfa7/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.111618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1281/2570] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:02 2022] DELETE /identity/v3/users/0fc060be899c467fab7abe95d09dfcb5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:03.127407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea9307a5b014be9bb7e2d2442ae7291, user_id=6f9142dbec9a4d9dbeba37ac9d85bfa7, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.130079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.145495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1290/2571] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:03 2022] PUT /identity/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291/users/6f9142dbec9a4d9dbeba37ac9d85bfa7/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:03.149509 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.214882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab92f4-e905-4e35-a7bd-12330fee1a31 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.236560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:03.278223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.278905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.279187 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.279506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.280230 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.281900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e876f93-35f7-4303-bd51-fb1f8fd48f23 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.292171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1282/2572] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:03.296291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.310457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55ff48-a341-408e-87d2-4fed571206fa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.317350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.318281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.318560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.318961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] PATH_INFO: `/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291/users/6f9142dbec9a4d9dbeba37ac9d85bfa7/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.321854 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea9307a5b014be9bb7e2d2442ae7291, user_id=6f9142dbec9a4d9dbeba37ac9d85bfa7, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.324747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.338799 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1291/2573] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:03 2022] PUT /identity/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291/users/6f9142dbec9a4d9dbeba37ac9d85bfa7/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:03.349197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.349620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.349929 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.379979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] MFA Rules not processed for user `6f9142dbec9a4d9dbeba37ac9d85bfa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:03.381345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.419082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.421143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.461841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1283/2574] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:03.473568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.475007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.495436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ab289-f462-4b37-9abc-a37b7a4cb3fa admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.500115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.500972 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.501258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.501535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] PATH_INFO: `/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.502739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465df59e06ab4b77bdd1970bab0ccbd2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.504712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.534561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791ee94-53ef-4976-93ec-e060e2120c8c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.538595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.539466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.539750 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.540023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.549919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.551346 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.554356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465df59e06ab4b77bdd1970bab0ccbd2', 'request_id': 'req-1aa7fb6b-ff91-4202-b346-f05f128473b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.560712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465df59e06ab4b77bdd1970bab0ccbd2', 'request_id': 'req-1aa7fb6b-ff91-4202-b346-f05f128473b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.560992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1284/2575] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:03.567702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465df59e06ab4b77bdd1970bab0ccbd2', 'request_id': 'req-1aa7fb6b-ff91-4202-b346-f05f128473b0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.581223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] The token cache is being invalidate because project 465df59e06ab4b77bdd1970bab0ccbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:03.581524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.582133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aa7fb6b-ff91-4202-b346-f05f128473b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.584189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1292/2576] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:03 2022] DELETE /identity/v3/projects/465df59e06ab4b77bdd1970bab0ccbd2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:03.590915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.591889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bef2-0c4d-4f7a-9efc-2b61e7acf8ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.592178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bef2-0c4d-4f7a-9efc-2b61e7acf8ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.592456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bef2-0c4d-4f7a-9efc-2b61e7acf8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.602349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc9fb-ae36-4fae-b6d6-0a4477fabca4 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.611719 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7645bef2-0c4d-4f7a-9efc-2b61e7acf8ce None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:03.612922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7645bef2-0c4d-4f7a-9efc-2b61e7acf8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.639900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:03.647338 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:03.661570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.662533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.662881 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.663164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] PATH_INFO: `/v3/users/6f156fbda63040eb9f75e8b97428fb55` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.664419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f156fbda63040eb9f75e8b97428fb55)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.666344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.677508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1293/2577] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:03.686288 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.686577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.686853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.705336 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:03.706560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.714497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f156fbda63040eb9f75e8b97428fb55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.718580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1294/2578] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:03.719405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f156fbda63040eb9f75e8b97428fb55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.719959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.730197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f156fbda63040eb9f75e8b97428fb55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.731070 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f156fbda63040eb9f75e8b97428fb55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.731430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57be0e-5d5d-4584-b392-e968b61596b4 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.736806 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.737468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.737750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.738027 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.738747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:03.739055 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.740712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.743942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-31045806-680e-41cb-9a24-46c248b98c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f156fbda63040eb9f75e8b97428fb55'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:03.753565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1295/2579] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:03.757771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.759015 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1285/2580] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:03 2022] DELETE /identity/v3/users/6f156fbda63040eb9f75e8b97428fb55 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:03.762273 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.763377 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.763920 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.764191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.776056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab63b3-e91f-42ee-8d3f-89c310d6b688 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.783126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:03.784339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.837994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.838565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.838864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.839115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.839835 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.841738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.847659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1286/2581] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:03.854008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.855387 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:03.872762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1296/2582] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:03.877421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.920680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86efd16-8859-4f82-9e86-ae8dacf264dd None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.941555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:03.948132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267e0a-1338-42b5-a61c-ea2e30d236dc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.953560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.954826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.954826 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.955217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.955981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.957911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.975136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.975830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:03.976114 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:03.976390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:03.979971 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1297/2583] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:03 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:03.984078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:03.988253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:03.989809 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:03.994118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f3-c7ca-4810-97d5-a17254ab6e3a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:03.999493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:03.999650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1287/2584] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:04.003931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.004217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.004492 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.005219 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.006819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.017324 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1298/2585] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:04.021215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.031671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536172-1220-4f89-9ec8-c0386110dd0b service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.036350 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.037007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.037291 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.037566 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.038284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.039932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c29925-210d-4d73-b4aa-49bde409a3b5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.051394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1288/2586] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:04.056115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.068601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc853fe-845a-4b05-b78d-db3eb98c60e6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.074005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.074911 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.075194 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.075525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] PATH_INFO: `/v3/projects/bc88adea07fe4264b496b0e273db7bca/users/51683eb30c494bb3a93910d66775d05c/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.088295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:04.092536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc88adea07fe4264b496b0e273db7bca, user_id=51683eb30c494bb3a93910d66775d05c, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.096613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.114721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1299/2587] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:04 2022] PUT /identity/v3/projects/bc88adea07fe4264b496b0e273db7bca/users/51683eb30c494bb3a93910d66775d05c/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:04.130274 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.130650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.131034 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.160574 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] MFA Rules not processed for user `51683eb30c494bb3a93910d66775d05c`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:04.161873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.229433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1289/2588] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:04.232377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.280913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.284294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.299422 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752fa9-72e6-4461-8f77-d4d2f9a0ae21 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.305103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.305796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.306075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.306422 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.307029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.307459 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19c5d0-316b-4a54-9667-7ce7009eda97 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.308964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.313414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.314349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.314632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.314905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] PATH_INFO: `/v3/projects/5ad2146a47134604b60d2ddbb8e8e1a1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.316149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad2146a47134604b60d2ddbb8e8e1a1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.318101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.336359 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1300/2589] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:04.340292 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.372193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad2146a47134604b60d2ddbb8e8e1a1', 'request_id': 'req-520fa30a-04ab-4e71-9813-c9fda4d07a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.379690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad2146a47134604b60d2ddbb8e8e1a1', 'request_id': 'req-520fa30a-04ab-4e71-9813-c9fda4d07a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.387654 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad2146a47134604b60d2ddbb8e8e1a1', 'request_id': 'req-520fa30a-04ab-4e71-9813-c9fda4d07a04'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.399194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] The token cache is being invalidate because project 5ad2146a47134604b60d2ddbb8e8e1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:04.399590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.400325 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.402074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1290/2590] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:04 2022] DELETE /identity/v3/projects/5ad2146a47134604b60d2ddbb8e8e1a1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:04.405430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e201-8051-4317-b93b-454b69cb6a15 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.406104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.407571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.440579 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:04.465375 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.466069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.466368 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.466619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.467351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.469302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.479464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa30a-04ab-4e71-9813-c9fda4d07a04 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.484813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.488151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.488439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.488716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] PATH_INFO: `/v3/users/6f9142dbec9a4d9dbeba37ac9d85bfa7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.489984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9142dbec9a4d9dbeba37ac9d85bfa7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.494393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1301/2591] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:04 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:04.494550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.496193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.498175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.512217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d12ac-150b-4c0b-9a8a-de146d6287d1 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.517963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.518636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.519424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.519424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.519986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.521641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.535270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1302/2592] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:04.536578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.541620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:04.548519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e92f-2742-4277-8cc9-0277a9f246f2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.548934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9142dbec9a4d9dbeba37ac9d85bfa7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.549704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9142dbec9a4d9dbeba37ac9d85bfa7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.555325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.556593 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.557209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.557413 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9142dbec9a4d9dbeba37ac9d85bfa7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.557545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] PATH_INFO: `/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac/users/82c4bd38e15048afac376f00c8da9fb1/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.559033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9142dbec9a4d9dbeba37ac9d85bfa7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.572381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9142dbec9a4d9dbeba37ac9d85bfa7'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:04.573473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01728bfa4ff4fa6994cfaa75281a5ac, user_id=82c4bd38e15048afac376f00c8da9fb1, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.576391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.588089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1291/2593] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:04 2022] DELETE /identity/v3/users/6f9142dbec9a4d9dbeba37ac9d85bfa7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:04.591161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1303/2594] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:04 2022] PUT /identity/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac/users/82c4bd38e15048afac376f00c8da9fb1/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:04.595334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.596758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.665253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b390f-e9e8-4bff-9fff-bca7528a12bf admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.670040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.670707 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.670991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.671298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.672073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.673679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dff81-fa20-4859-a433-e834eb4c8729 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.683941 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1292/2595] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:04.688245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.756784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec8d0f-89e2-4621-8e8e-39b587422ecc admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.761964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.762858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.763141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.763703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] PATH_INFO: `/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac/users/82c4bd38e15048afac376f00c8da9fb1/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.765725 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01728bfa4ff4fa6994cfaa75281a5ac, user_id=82c4bd38e15048afac376f00c8da9fb1, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.768687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.779622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1304/2596] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:04 2022] PUT /identity/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac/users/82c4bd38e15048afac376f00c8da9fb1/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:04.789568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.789932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.790280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.818849 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] MFA Rules not processed for user `82c4bd38e15048afac376f00c8da9fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:04.820105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.853807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:04.877936 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.879838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.903464 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1293/2597] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:04.914875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:04.916270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:04.956523 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96813b9a-d98f-4110-8ed0-a861aa825f59 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.961171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.962053 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.962334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.962621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] PATH_INFO: `/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.963817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea9307a5b014be9bb7e2d2442ae7291)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.965720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:04.974981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7615-6f34-4d7d-9252-29d5ee57808d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:04.978932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:04.979613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:04.979894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:04.980168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:04.989462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:04.990889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc361e0-fd28-476b-b9c0-7759d3e50a9f service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.001560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1294/2598] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:05.015001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea9307a5b014be9bb7e2d2442ae7291', 'request_id': 'req-719f048b-adf1-4846-a41f-40fb6a335a83'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.020547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea9307a5b014be9bb7e2d2442ae7291', 'request_id': 'req-719f048b-adf1-4846-a41f-40fb6a335a83'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.027402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea9307a5b014be9bb7e2d2442ae7291', 'request_id': 'req-719f048b-adf1-4846-a41f-40fb6a335a83'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.047852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] The token cache is being invalidate because project 1ea9307a5b014be9bb7e2d2442ae7291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:05.048159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.048823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.050411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1305/2599] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:04 2022] DELETE /identity/v3/projects/1ea9307a5b014be9bb7e2d2442ae7291 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:05.055911 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff6fd-ab06-48a8-928d-c3c7713fe1b9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.056204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff6fd-ab06-48a8-928d-c3c7713fe1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.056484 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff6fd-ab06-48a8-928d-c3c7713fe1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.072962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.074902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.076741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1b0ff6fd-ab06-48a8-928d-c3c7713fe1b9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:05.077957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1b0ff6fd-ab06-48a8-928d-c3c7713fe1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.080817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:05.139982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1295/2600] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:05.142762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f048b-adf1-4846-a41f-40fb6a335a83 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.147485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.148355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.148639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.148916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] PATH_INFO: `/v3/users/51683eb30c494bb3a93910d66775d05c` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.149489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.149780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.150064 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.150256 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51683eb30c494bb3a93910d66775d05c)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.152267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.169412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:05.170626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.179357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1296/2601] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:05.183731 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.193874 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602a0b3-7d7c-4de2-93b1-582c34993dfc None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.198796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.199475 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.199751 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.200020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.200733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:05.201033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.201239 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51683eb30c494bb3a93910d66775d05c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.202650 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.208437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51683eb30c494bb3a93910d66775d05c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.209124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51683eb30c494bb3a93910d66775d05c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.215460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1297/2602] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:05.220657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51683eb30c494bb3a93910d66775d05c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.220787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.221298 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9410f4e3-4d94-4512-9cc0-bdbd8cb91182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51683eb30c494bb3a93910d66775d05c'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.222420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.235926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857483c-4177-4272-aa25-baef46a72a71 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.239773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1306/2603] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:05 2022] DELETE /identity/v3/users/51683eb30c494bb3a93910d66775d05c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:05.240556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.241200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.241467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.241762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.243036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.244372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.244686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.245242 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.245380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.264557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:05.266482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.273224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1298/2604] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:05.277327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.310641 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:05.330766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1307/2605] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:05.340437 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.341889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.342083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aea39d-8cd6-4230-aaba-29f738859a58 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.347359 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.348027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.348308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.348999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.349262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.351191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.372287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1299/2606] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:05.376394 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.376824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:05.386565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b7114-ac12-45cf-924c-c9413d1085ad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.392542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.393131 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.393408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.393679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.394397 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.396030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.405777 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1300/2607] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:05.410346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7304e-f38c-4d1f-85f0-4da18847b22f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.410598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.420284 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce25a4-b01c-482f-8cbd-13a0b290e2de admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.425045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.425717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.425998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.426271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.426996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.428730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.439302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1301/2608] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:05.443260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.452722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51723-7305-40e4-877f-971a11d31d04 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.457993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.458825 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.459110 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.459993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] PATH_INFO: `/v3/projects/6165017cb8ca4588bc889c917e95af22/users/334ed409322a45df8ce693bc37069831/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.462170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.462888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.463161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.463470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.473078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.474532 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bfcde-8f08-42f9-8f3d-82389d14c053 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.475794 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6165017cb8ca4588bc889c917e95af22, user_id=334ed409322a45df8ce693bc37069831, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.478590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.484759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1308/2609] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:05.494760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1302/2610] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:05 2022] PUT /identity/v3/projects/6165017cb8ca4588bc889c917e95af22/users/334ed409322a45df8ce693bc37069831/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:05.505654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.505948 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.506231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.534991 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] MFA Rules not processed for user `334ed409322a45df8ce693bc37069831`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:05.536271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.604376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1309/2611] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:05.608901 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.677366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ffd19-fbe0-492b-9ac9-cd387b5b3b0b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.681591 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.682243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.682523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.682798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.683520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.685410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.711186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1303/2612] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:05.715111 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.731765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64876bf-8abf-48f2-a973-0aed64165f63 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.768466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.770573 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.806867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.807575 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.807868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.808147 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.808849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.810796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.826516 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:05.835677 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1310/2613] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:05 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:05.836514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.845162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c640-33be-44bc-88f1-d5b1f156287e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.846582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdbc76-83de-4a0a-a202-1fc468f437c4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.850235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.851081 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.851225 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.851530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.851751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.851860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] PATH_INFO: `/v3/projects/bc88adea07fe4264b496b0e273db7bca` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.852032 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.852309 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.852983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc88adea07fe4264b496b0e273db7bca)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.853223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.854798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.855402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.857415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:05.865823 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1311/2614] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:05.870141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.879954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab1930-6a72-4bc8-807e-17bad6ef4329 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.884629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.885642 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.885939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.886216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] PATH_INFO: `/v3/projects/33e214742a8947b096c40e196a76e107/users/c657b3692ea84ce5b1442591d14a5463/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.902040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e214742a8947b096c40e196a76e107, user_id=c657b3692ea84ce5b1442591d14a5463, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.904722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:05.905891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc88adea07fe4264b496b0e273db7bca', 'request_id': 'req-12023b99-d645-40ae-a255-60bf23bebb3b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.912752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc88adea07fe4264b496b0e273db7bca', 'request_id': 'req-12023b99-d645-40ae-a255-60bf23bebb3b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.920455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1312/2615] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:05 2022] PUT /identity/v3/projects/33e214742a8947b096c40e196a76e107/users/c657b3692ea84ce5b1442591d14a5463/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:05.924126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc88adea07fe4264b496b0e273db7bca', 'request_id': 'req-12023b99-d645-40ae-a255-60bf23bebb3b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.924756 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.936541 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] The token cache is being invalidate because project bc88adea07fe4264b496b0e273db7bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:05.936753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.937993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:05.939688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1304/2616] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:05 2022] DELETE /identity/v3/projects/bc88adea07fe4264b496b0e273db7bca => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:05.943369 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:05.944709 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:05.987629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353e415-d390-45fb-90dc-8e9e37209d72 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:05.992193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:05.992852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:05.993135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:05.993412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:05.994115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:05.995748 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.005907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1313/2617] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:06.009886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.011527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.013143 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023b99-d645-40ae-a255-60bf23bebb3b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.018768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.019679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.019968 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.020243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] PATH_INFO: `/v3/users/82c4bd38e15048afac376f00c8da9fb1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.021430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c4bd38e15048afac376f00c8da9fb1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.023392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.024762 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8e5dc-d750-4409-9e78-5a6183b5a8eb admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.029276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.030162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.030441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.030713 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] PATH_INFO: `/v3/projects/33e214742a8947b096c40e196a76e107/users/c657b3692ea84ce5b1442591d14a5463/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.032690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e214742a8947b096c40e196a76e107, user_id=c657b3692ea84ce5b1442591d14a5463, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.035351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630c542-a859-4916-b8d2-5653ccdd6635 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.045462 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1314/2618] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:06 2022] PUT /identity/v3/projects/33e214742a8947b096c40e196a76e107/users/c657b3692ea84ce5b1442591d14a5463/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:06.057105 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.057472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.057750 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.074068 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c4bd38e15048afac376f00c8da9fb1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.074737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c4bd38e15048afac376f00c8da9fb1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.082343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c4bd38e15048afac376f00c8da9fb1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.083445 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c4bd38e15048afac376f00c8da9fb1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.084812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] MFA Rules not processed for user `c657b3692ea84ce5b1442591d14a5463`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:06.086003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.095933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c4bd38e15048afac376f00c8da9fb1'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.111846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1305/2619] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:05 2022] DELETE /identity/v3/users/82c4bd38e15048afac376f00c8da9fb1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:06.167572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1315/2620] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:06.176117 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.177558 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.189906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:06.241366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cae48-a426-4837-93c4-25b2bec967f0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.245895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.246586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.246928 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.247194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.315964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.317444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.327248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1306/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:06.391079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.392747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.431170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.436108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:06.440635 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bb997-8a9e-4f16-81bc-69872d8c78f3 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.444800 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.445418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.445701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.445969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.447786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.467262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c8ab-fc62-41f6-839e-df00b53ab1d8 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.511555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.513005 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd1d7e-9a0c-4108-9fbb-4d8719357398 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.523664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1307/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:06.530378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.531172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.531606 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.531882 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] PATH_INFO: `/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.533052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01728bfa4ff4fa6994cfaa75281a5ac)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.535228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.583578 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01728bfa4ff4fa6994cfaa75281a5ac', 'request_id': 'req-eafdc707-f4f4-48c9-92a9-c579ac51235e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.589167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01728bfa4ff4fa6994cfaa75281a5ac', 'request_id': 'req-eafdc707-f4f4-48c9-92a9-c579ac51235e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.595376 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01728bfa4ff4fa6994cfaa75281a5ac', 'request_id': 'req-eafdc707-f4f4-48c9-92a9-c579ac51235e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.608134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] The token cache is being invalidate because project f01728bfa4ff4fa6994cfaa75281a5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:06.608425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.609064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-eafdc707-f4f4-48c9-92a9-c579ac51235e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:06.610716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1316/2623] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:06 2022] DELETE /identity/v3/projects/f01728bfa4ff4fa6994cfaa75281a5ac => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:06.615626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.615915 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.616194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.635072 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:06.636314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.694766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=100 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:06.705821 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1308/2624] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:06.714723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.715163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.715558 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.736391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:06.737817 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:06.746304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1317/2625] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:06.751434 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.774857 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab99afd-fe98-49cf-9e45-a7c0af57ee5b None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.785848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.787392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.787685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.792667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.792667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:06.792667 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.793831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.814712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1309/2626] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:06.819051 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.828961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30c638-bdaf-49f5-b265-f89cf451ffd7 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.833998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.834692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.835009 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.835307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.836010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.837909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.867909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1318/2627] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:06 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:06.869051 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.880017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044d1d6-fd2e-41f6-95ca-9bfd5a6e4eb5 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.937107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.939231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.939539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.939813 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.940506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.942429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.962183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1310/2628] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:06 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:06.966442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:06.976758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92089db-05cd-4ab1-aac0-020c5369882d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:06.981366 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:06.982153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:06.982439 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:06.982720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:06.983480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:06.985681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:06.995935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1319/2629] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:07.000450 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.010662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec13258-3726-41cf-aefb-9c172e881b54 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.015536 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.016282 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.016567 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.016847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.017594 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.019182 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e565d-268a-4803-9ab2-a66e96d37f42 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.028865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1311/2630] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:07.032861 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.043157 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc52d-6f4e-4a95-8a9d-e7325e7c6458 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.048457 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.049330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.049613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.049891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] PATH_INFO: `/v3/projects/dc0a4bc3d5ce49fab99859e537e7e1cd/users/fae9317803b340b6b6a22b46892852b9/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.064718 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0a4bc3d5ce49fab99859e537e7e1cd, user_id=fae9317803b340b6b6a22b46892852b9, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.067463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.080547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1320/2631] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:07 2022] PUT /identity/v3/projects/dc0a4bc3d5ce49fab99859e537e7e1cd/users/fae9317803b340b6b6a22b46892852b9/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:07.091258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.091555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.091879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.119153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] MFA Rules not processed for user `fae9317803b340b6b6a22b46892852b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:07.120399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:07.184265 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1312/2632] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:07.189108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.198429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:07.258096 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef40f4e-d25c-46fe-8f18-75d496fb3cc5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.262763 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.263690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.263980 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.264255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.264949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.266896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.291827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1321/2633] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:07 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:07.295859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.306645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b508f-8e5c-43e5-bbc6-f255a57c0d2f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.353305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:07.363091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.363788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.364074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.364354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.365038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.366979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.390367 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1313/2634] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:07 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:07.392283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.402495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061738c0-0170-44d1-8b6b-eec04741fe9e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.407386 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.408037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.408317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.408588 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.409299 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.410885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.421125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1322/2635] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:07.425298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.435009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf761fb9-26c2-433f-b7ec-d3651995adc7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.440510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.441375 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.441655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.441955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] PATH_INFO: `/v3/projects/f573058d3a784f778a58723a36a46da5/users/45dd65b2aba14d619b7245c6d3ce2667/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.456797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f573058d3a784f778a58723a36a46da5, user_id=45dd65b2aba14d619b7245c6d3ce2667, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.459957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.472786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1314/2636] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:07 2022] PUT /identity/v3/projects/f573058d3a784f778a58723a36a46da5/users/45dd65b2aba14d619b7245c6d3ce2667/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:07.476644 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.486657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32eeca-b55d-4000-8a67-6dafe289e60e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.543828 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.544499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.544784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.545061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.545785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.547931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db14849-6e69-4580-9d48-6006949fe85f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.557480 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1323/2637] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:07.561804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.572732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90883-4422-43c8-84f9-b21a097f8fd2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.577356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.578279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.578636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.578924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] PATH_INFO: `/v3/projects/f573058d3a784f778a58723a36a46da5/users/45dd65b2aba14d619b7245c6d3ce2667/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.580935 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f573058d3a784f778a58723a36a46da5, user_id=45dd65b2aba14d619b7245c6d3ce2667, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.583636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.593758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1315/2638] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:07 2022] PUT /identity/v3/projects/f573058d3a784f778a58723a36a46da5/users/45dd65b2aba14d619b7245c6d3ce2667/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:07.603833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.604130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.604410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.631270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] MFA Rules not processed for user `45dd65b2aba14d619b7245c6d3ce2667`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:07.632487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:07.699190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1324/2639] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:07.710432 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:07.712227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:07.772446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205c678-8b4d-4cb0-96c1-f127299507ed admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:07.776045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:07.776672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:07.776950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:07.777229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:07.786237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:07.787692 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd57a9-7813-4d60-864b-5da9d1797e80 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:07.797251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1316/2640] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:08.367310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:08.371654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:08.432887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbaad2-2622-4468-94c4-d1517bc23487 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:08.438140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:08.439133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:08.439442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:08.439717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] PATH_INFO: `/v3/users/fae9317803b340b6b6a22b46892852b9` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:08.440908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae9317803b340b6b6a22b46892852b9)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:08.442849 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:08.493183 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae9317803b340b6b6a22b46892852b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:08.500612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae9317803b340b6b6a22b46892852b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:08.501350 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae9317803b340b6b6a22b46892852b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:08.508485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae9317803b340b6b6a22b46892852b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:08.509356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae9317803b340b6b6a22b46892852b9'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:08.526678 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1325/2641] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:08 2022] DELETE /identity/v3/users/fae9317803b340b6b6a22b46892852b9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:08.531078 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:08.531462 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:08.531796 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:08.550490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:08.551744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:08.610560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1317/2642] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:08.620058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:08.621443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:08.680072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9356a59-bf3e-440d-8b5f-2b9545118e45 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:08.683656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:08.684300 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:08.684584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:08.684858 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:08.693775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:08.695208 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:08.704692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1326/2643] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:08.977812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:08.979220 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:08.993184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f466d4-6e38-4863-b7ee-09ad7fc42ed8 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:08.997808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:08.998665 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:08.998949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:08.999223 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] PATH_INFO: `/v3/projects/dc0a4bc3d5ce49fab99859e537e7e1cd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.000998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0a4bc3d5ce49fab99859e537e7e1cd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.003147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.047304 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:09.051207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0a4bc3d5ce49fab99859e537e7e1cd', 'request_id': 'req-13357fce-c30a-4a66-b4ff-99c304069615'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.058917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0a4bc3d5ce49fab99859e537e7e1cd', 'request_id': 'req-13357fce-c30a-4a66-b4ff-99c304069615'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.065807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0a4bc3d5ce49fab99859e537e7e1cd', 'request_id': 'req-13357fce-c30a-4a66-b4ff-99c304069615'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.076020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] The token cache is being invalidate because project dc0a4bc3d5ce49fab99859e537e7e1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:09.076308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.076918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.079031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1318/2644] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:08 2022] DELETE /identity/v3/projects/dc0a4bc3d5ce49fab99859e537e7e1cd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:09.083000 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.092557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d04c2-7062-49a4-b6a0-d606f24c6efd service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.116215 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.118083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.118356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:09.159143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.160072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.160353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.160624 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] PATH_INFO: `/v3/users/45dd65b2aba14d619b7245c6d3ce2667` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.161802 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45dd65b2aba14d619b7245c6d3ce2667)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.163790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.182823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357fce-c30a-4a66-b4ff-99c304069615 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.187688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.188551 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.188831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.189105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] PATH_INFO: `/v3/users/334ed409322a45df8ce693bc37069831` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.190314 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334ed409322a45df8ce693bc37069831)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.192390 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.211573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45dd65b2aba14d619b7245c6d3ce2667'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.218865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45dd65b2aba14d619b7245c6d3ce2667'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.219651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45dd65b2aba14d619b7245c6d3ce2667'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.227057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45dd65b2aba14d619b7245c6d3ce2667'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.227768 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9c46278d-f7cf-4f7e-95ca-c35e3650c9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45dd65b2aba14d619b7245c6d3ce2667'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.244133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334ed409322a45df8ce693bc37069831'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.245092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334ed409322a45df8ce693bc37069831'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.247145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1327/2645] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:09 2022] DELETE /identity/v3/users/45dd65b2aba14d619b7245c6d3ce2667 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:09.253964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334ed409322a45df8ce693bc37069831'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.254660 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334ed409322a45df8ce693bc37069831'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.267735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334ed409322a45df8ce693bc37069831'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.283335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1319/2646] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:09 2022] DELETE /identity/v3/users/334ed409322a45df8ce693bc37069831 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:09.287977 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.288204 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.288514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.306702 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:09.307945 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.369085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1328/2647] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:09.379263 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.380863 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.442080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45411d9c-de4b-43ce-a17c-41f74a56f7f3 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.445757 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.446400 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.446680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.446955 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.454246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:09.456954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.459682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc057c-4363-4856-b9ee-9dd4bf48c7a9 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.475862 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1320/2648] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:09.599360 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.600847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.613549 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1b06f-302e-45ce-b634-53faf07de186 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.621955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.623538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.623827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.624458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] PATH_INFO: `/v3/projects/f573058d3a784f778a58723a36a46da5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.626225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f573058d3a784f778a58723a36a46da5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.628209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.673280 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:09.681741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f573058d3a784f778a58723a36a46da5', 'request_id': 'req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.697412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f573058d3a784f778a58723a36a46da5', 'request_id': 'req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.704346 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f573058d3a784f778a58723a36a46da5', 'request_id': 'req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.717597 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] The token cache is being invalidate because project f573058d3a784f778a58723a36a46da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:09.717887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.719069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.720815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1329/2649] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:09 2022] DELETE /identity/v3/projects/f573058d3a784f778a58723a36a46da5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:09.725628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22923b8a-83b6-4d44-9a88-bee4f491fea9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.725986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22923b8a-83b6-4d44-9a88-bee4f491fea9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.726356 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22923b8a-83b6-4d44-9a88-bee4f491fea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.751105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-22923b8a-83b6-4d44-9a88-bee4f491fea9 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:09.752381 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-22923b8a-83b6-4d44-9a88-bee4f491fea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.820669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.822203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.831515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1321/2650] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:09.837670 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.837800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea35-3ef6-4221-85fb-83ba98e2b63a admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.837990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.838235 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.843117 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.844726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.845206 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.845316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] PATH_INFO: `/v3/projects/6165017cb8ca4588bc889c917e95af22` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.850881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6165017cb8ca4588bc889c917e95af22)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.853264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.869453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:09.870662 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.878763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1322/2651] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:09.882961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.893171 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e988d1-707a-4664-86f7-5c7039ea361e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.907583 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6165017cb8ca4588bc889c917e95af22', 'request_id': 'req-8002d5b3-bba2-451f-8b90-eb3880c5cc45'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.913092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6165017cb8ca4588bc889c917e95af22', 'request_id': 'req-8002d5b3-bba2-451f-8b90-eb3880c5cc45'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.919942 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6165017cb8ca4588bc889c917e95af22', 'request_id': 'req-8002d5b3-bba2-451f-8b90-eb3880c5cc45'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.934316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] The token cache is being invalidate because project 6165017cb8ca4588bc889c917e95af22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:09.934607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.935285 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:09.936946 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1330/2652] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:09 2022] DELETE /identity/v3/projects/6165017cb8ca4588bc889c917e95af22 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:09.941339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.942783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.948961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.949618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.949897 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.950168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.950884 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:09.951194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.953341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:09.964952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1323/2653] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:09.969774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:09.971886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:09.984177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0bcd6-ee8d-454c-bf11-0e3cd00fa319 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:09.988910 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:09.989553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:09.989831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:09.990104 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:09.990795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:09.992721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.009502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002d5b3-bba2-451f-8b90-eb3880c5cc45 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.014864 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.016291 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.016564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.016837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] PATH_INFO: `/v3/users/c657b3692ea84ce5b1442591d14a5463` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.018042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c657b3692ea84ce5b1442591d14a5463)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.020011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.020596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1324/2654] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:09 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:10.024531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.069005 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c657b3692ea84ce5b1442591d14a5463'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.073512 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:10.076931 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c657b3692ea84ce5b1442591d14a5463'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.078536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c657b3692ea84ce5b1442591d14a5463'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.086478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c657b3692ea84ce5b1442591d14a5463'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.087103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c657b3692ea84ce5b1442591d14a5463'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.088651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006f8cf-1e35-47c8-bc27-947de9f37260 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.093481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.094202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.094490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.094764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.095511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.097440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.104085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:10.104686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1331/2655] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:09 2022] DELETE /identity/v3/users/c657b3692ea84ce5b1442591d14a5463 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:10.119669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1325/2656] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:10.123739 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.125260 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.199446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc339-cee6-463f-8d4f-26cdcebe039d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.204197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.204869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.205149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.205421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.206172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.207820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.218954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1332/2657] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:10.222918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.233074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5080c2-021c-42c7-ae55-250c220fd459 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.238478 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.239146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.239468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.239868 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.240598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.242278 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9cf40-f2a5-4167-9faa-0342a61197db admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.254451 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1326/2658] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:10.260152 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.271706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2151d-5b5a-4a19-8c9f-4512b7cacba0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.276374 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.277267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.277542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.277814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] PATH_INFO: `/v3/projects/0b94affae5e947e9aa69f962fdb4c7d6/users/dd649aa6d5f7444ab583fc28f9e273da/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.293641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b94affae5e947e9aa69f962fdb4c7d6, user_id=dd649aa6d5f7444ab583fc28f9e273da, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.296364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.314459 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1333/2659] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:10 2022] PUT /identity/v3/projects/0b94affae5e947e9aa69f962fdb4c7d6/users/dd649aa6d5f7444ab583fc28f9e273da/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:10.319615 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.319909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.320190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.337744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:10.338966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.401228 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.402918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.404542 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1327/2660] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:10.414543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.415979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.422692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19a5e7-d4af-4d21-aa2c-094e2068a2ad admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.428737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.429605 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.429891 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.430169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] PATH_INFO: `/v3/projects/33e214742a8947b096c40e196a76e107` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.431358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e214742a8947b096c40e196a76e107)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.433293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.478664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a09ed-5249-4ae3-bdc6-e545e6aa0baf None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.493898 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e214742a8947b096c40e196a76e107', 'request_id': 'req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.499609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e214742a8947b096c40e196a76e107', 'request_id': 'req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.506609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e214742a8947b096c40e196a76e107', 'request_id': 'req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.521159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] The token cache is being invalidate because project 33e214742a8947b096c40e196a76e107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:10.521581 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.522230 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-c7c5d9c8-5e61-4458-8b3e-16b92b7edef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:10.523831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1334/2661] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:10 2022] DELETE /identity/v3/projects/33e214742a8947b096c40e196a76e107 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:10.528430 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8999-e44c-48e9-b68c-382335d8131b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.528723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8999-e44c-48e9-b68c-382335d8131b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.529003 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8999-e44c-48e9-b68c-382335d8131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.529260 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.529902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.530236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.530516 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.532428 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.549088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f61c8999-e44c-48e9-b68c-382335d8131b None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:10.550287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f61c8999-e44c-48e9-b68c-382335d8131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.570023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:10.586004 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:10.603722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.605175 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.614244 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1335/2662] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:10.615195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1328/2663] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:10.623404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.623688 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.623970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.643493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:10.644723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:10.652780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1336/2664] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:10.657293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.667474 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175617c-97b3-4cd8-9b03-a64cb665853f service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.672806 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.673584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.673937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.674290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.675095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:10.675718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.677430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.691102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1329/2665] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 03:26:10.695491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.705393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9246aef-e518-4710-817f-f42f9e09a5b5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.710160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.710820 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.711103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.711411 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.712121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.714052 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.746162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1337/2666] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:10.747774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.759052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d545486-81c4-4801-acd9-ed6a7fed0b6d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.824371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.825098 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.825385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.825655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.826481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.828463 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.851828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1330/2667] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:10.853352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.865029 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1aa256-bfbe-4d1f-86b8-2c772b610363 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.870824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.871540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.871844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.872214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.872944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.874652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.886176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1338/2668] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:10.890720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.900891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ef4a-2e74-4f1e-a86e-1272a9f4b929 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.905718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.906908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.907180 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.907472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.908190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.909787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69072b-4de0-4e34-a6fe-17cd60c56230 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.920519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1331/2669] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:10.925310 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:10.936238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cce18-0272-4ea2-b512-9810aa00b1ba admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:10.941612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:10.942556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.942955 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.943146 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] PATH_INFO: `/v3/projects/8fe938eb52d1431fb8163fba3a49f8f6/users/829d4df5b22e4b3e8da932cb25a40efd/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:10.963304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe938eb52d1431fb8163fba3a49f8f6, user_id=829d4df5b22e4b3e8da932cb25a40efd, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:10.966213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:10.980757 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1339/2670] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:10 2022] PUT /identity/v3/projects/8fe938eb52d1431fb8163fba3a49f8f6/users/829d4df5b22e4b3e8da932cb25a40efd/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:10.991879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e10c37-ffa9-4329-acb1-322627fc4b3a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:10.992438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e10c37-ffa9-4329-acb1-322627fc4b3a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:10.992438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e10c37-ffa9-4329-acb1-322627fc4b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.020771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-35e10c37-ffa9-4329-acb1-322627fc4b3a None None] MFA Rules not processed for user `829d4df5b22e4b3e8da932cb25a40efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:11.021971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-35e10c37-ffa9-4329-acb1-322627fc4b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:11.097921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1332/2671] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:11.111358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.113149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:11.180522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b0ab7-1fb5-429d-8300-8b77805b5339 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:11.185463 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b48b2079573456c93a286b7d799e950', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:11.185660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.185893 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.186226 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.197880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:11.199569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:11.209704 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1340/2672] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:11.532792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.533396 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.533705 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.571842 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] MFA Rules not processed for user `dd649aa6d5f7444ab583fc28f9e273da`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:11.573219 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:11.609939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:11.660963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1333/2673] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:11.664957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.666538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:11.757646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cf47c-87ff-4612-b4c1-564cc4b5e019 service glance] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:11.763231 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:11.764181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.764467 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.764743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.766121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:11.768200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:11.777087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:11.797454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1341/2674] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:11 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:11.801799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.812942 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e4a08-491d-4f03-afc2-bc4fa264da49 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:11.868372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:11.869038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.869321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.869596 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.870287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:11.872248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:11.896637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1334/2675] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:11 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:11.897837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.908724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b05dc6-d155-4616-afbd-9e33fcf64dd4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:11.913515 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:11.914121 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.914404 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.914715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.915544 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:11.917119 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:11.928472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1342/2676] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:11.932480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.944065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26dfe-6e3c-40d9-943e-75d09a0086d1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:11.948797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:11.949739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:11.950023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:11.950332 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] PATH_INFO: `/v3/projects/d3226fabce5f4db99b5cec98ee7c6900/users/32c5ef1f5c8d4258949ce4f08e560868/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:11.965418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3226fabce5f4db99b5cec98ee7c6900, user_id=32c5ef1f5c8d4258949ce4f08e560868, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:11.968146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:11.982067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1335/2677] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:11 2022] PUT /identity/v3/projects/d3226fabce5f4db99b5cec98ee7c6900/users/32c5ef1f5c8d4258949ce4f08e560868/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:11.986902 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:11.999522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a870d9-c087-4818-aad7-55f99a4961e3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:12.059460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:12.060129 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:12.060408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:12.060681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:12.061398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:12.063131 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc495728-6d58-4435-8dfc-4c14f852dc45 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:12.074721 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1343/2678] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:12.079506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:12.090042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a98683-93d5-4a45-a42e-ddc84587ee53 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:12.095398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:12.096270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:12.096552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:12.096828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] PATH_INFO: `/v3/projects/d3226fabce5f4db99b5cec98ee7c6900/users/32c5ef1f5c8d4258949ce4f08e560868/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:12.098784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3226fabce5f4db99b5cec98ee7c6900, user_id=32c5ef1f5c8d4258949ce4f08e560868, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:12.101446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:12.112679 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1336/2679] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:12 2022] PUT /identity/v3/projects/d3226fabce5f4db99b5cec98ee7c6900/users/32c5ef1f5c8d4258949ce4f08e560868/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:12.751668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:12.752091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:12.752249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:12.785711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] MFA Rules not processed for user `32c5ef1f5c8d4258949ce4f08e560868`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:12.787156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:12.871496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1344/2680] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:12.882690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:12.884405 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:12.946986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60332ee7-83c0-4d00-ab70-46e63250efe9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:12.950805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:12.951513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:12.951829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:12.952106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:12.986823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:13.014101 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:13.015732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:13.024887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1337/2681] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:26:14.383453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:14.393289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8a11-4982-4ae4-a655-39df4dc75bc5 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:14.396803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:14.397442 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:14.397727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:14.398004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:14.407295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:14.408804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:14.419670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1345/2682] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:23.020730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=11 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:24.415060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:27.392546 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:27.466779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1043-5a4e-4ce7-a246-931ad21b83ec service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:27.471539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:27.472408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:27.472687 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:27.472960 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] PATH_INFO: `/v3/users/dd649aa6d5f7444ab583fc28f9e273da` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:27.475345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd649aa6d5f7444ab583fc28f9e273da)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:27.477653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:27.541932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd649aa6d5f7444ab583fc28f9e273da'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.542680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd649aa6d5f7444ab583fc28f9e273da'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.551771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd649aa6d5f7444ab583fc28f9e273da'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.553003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd649aa6d5f7444ab583fc28f9e273da'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.566520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd649aa6d5f7444ab583fc28f9e273da'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.582466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1338/2683] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:27 2022] DELETE /identity/v3/users/dd649aa6d5f7444ab583fc28f9e273da => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:27.770510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:27.785683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c75e9-6190-482a-a329-c50d3cafd8ad service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:27.869841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:27.871080 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:27.873519 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:27.873710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] PATH_INFO: `/v3/projects/0b94affae5e947e9aa69f962fdb4c7d6` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:27.876023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b94affae5e947e9aa69f962fdb4c7d6)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:27.878267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:27.949276 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b94affae5e947e9aa69f962fdb4c7d6', 'request_id': 'req-f67c41b2-556c-4e37-8dee-5e73771ef9c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.956073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b94affae5e947e9aa69f962fdb4c7d6', 'request_id': 'req-f67c41b2-556c-4e37-8dee-5e73771ef9c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.964846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b94affae5e947e9aa69f962fdb4c7d6', 'request_id': 'req-f67c41b2-556c-4e37-8dee-5e73771ef9c0'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.979803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] The token cache is being invalidate because project 0b94affae5e947e9aa69f962fdb4c7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:27.980127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.981022 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:27.982662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1346/2684] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:27 2022] DELETE /identity/v3/projects/0b94affae5e947e9aa69f962fdb4c7d6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:27.987231 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:27.988710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:28.004159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da275c-1651-4fbf-8dc7-e7abeb797503 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.067029 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.068644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.068926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.069205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] PATH_INFO: `/v3/users/32c5ef1f5c8d4258949ce4f08e560868` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.070391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c5ef1f5c8d4258949ce4f08e560868)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.072363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.094988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:28.124186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c5ef1f5c8d4258949ce4f08e560868'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.124829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c5ef1f5c8d4258949ce4f08e560868'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.132850 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c5ef1f5c8d4258949ce4f08e560868'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.133489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c5ef1f5c8d4258949ce4f08e560868'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.146581 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3c8e8205-bffe-4dcd-8ccf-207ee52ddca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c5ef1f5c8d4258949ce4f08e560868'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.159844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1339/2685] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:27 2022] DELETE /identity/v3/users/32c5ef1f5c8d4258949ce4f08e560868 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:28.339223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.340962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:28.355033 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c41b2-556c-4e37-8dee-5e73771ef9c0 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.376798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757b9a-c142-475a-b44c-e1c91a69fbac None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.377083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757b9a-c142-475a-b44c-e1c91a69fbac None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.377354 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757b9a-c142-475a-b44c-e1c91a69fbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.399176 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e757b9a-c142-475a-b44c-e1c91a69fbac None None] MFA Rules not processed for user `9e41cedaabc54e408cdddee2e5cb140d`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:28.400437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e757b9a-c142-475a-b44c-e1c91a69fbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:28.434871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.435809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.436094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.436377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] PATH_INFO: `/v3/projects/d3226fabce5f4db99b5cec98ee7c6900` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.437570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3226fabce5f4db99b5cec98ee7c6900)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.439576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.456861 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1340/2686] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:28.475847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:28.490484 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3226fabce5f4db99b5cec98ee7c6900', 'request_id': 'req-52443282-f02a-4603-a64d-997fe1c3d74a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.496732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3226fabce5f4db99b5cec98ee7c6900', 'request_id': 'req-52443282-f02a-4603-a64d-997fe1c3d74a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.504464 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3226fabce5f4db99b5cec98ee7c6900', 'request_id': 'req-52443282-f02a-4603-a64d-997fe1c3d74a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.520176 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] The token cache is being invalidate because project d3226fabce5f4db99b5cec98ee7c6900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:28.520495 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.521685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-52443282-f02a-4603-a64d-997fe1c3d74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:28.523398 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1347/2687] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:28 2022] DELETE /identity/v3/projects/d3226fabce5f4db99b5cec98ee7c6900 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:28.528290 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.528585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.528864 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.547486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:28.548710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:28.612525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1341/2688] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:28.621342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.621635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.621915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.641703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:28.643010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:28.651620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1348/2689] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:28.655905 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.666147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c2c9-cc31-4eba-9d4b-f32543653c2e None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.670756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.671435 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.671714 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.671981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.672704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:28.673003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.674597 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.689720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1342/2690] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:26:28.690790 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.700402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba6114-e2bf-4e5c-a721-52cdaceb5c2a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.705064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.705717 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.705999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.706274 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.706966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.709797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.745538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1349/2691] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:28.746625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.757226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741a34-9bb6-4f52-9a64-a440bc8ca3c8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.787190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:28.816246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.816944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.817251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.818437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.818775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.820733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.841186 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1343/2692] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:28.845250 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.856073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a5c4-0b50-4a34-8be5-d7939f83aa1e admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.860620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.861282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.861568 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.861842 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.862560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.864171 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.875082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1350/2693] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:28.879040 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.890672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b20998-0aef-4493-9a18-ce5ba3df216b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.896146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.896982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.897337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.897685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.898525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.900243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c3a19-41d9-4bde-b6b2-bcf09b432bc6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.910598 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1344/2694] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:28.916082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:28.929019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e835e9-3544-4ce3-8a49-9eb40d114c3d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:28.934808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:28.936145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.936591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.937028 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] PATH_INFO: `/v3/projects/6e68e2ea99614c628e140a6c59646315/users/208fefff0fed4fda80c40b1240b16c8f/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:28.954270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e68e2ea99614c628e140a6c59646315, user_id=208fefff0fed4fda80c40b1240b16c8f, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:28.957130 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:28.978188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1351/2695] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:28 2022] PUT /identity/v3/projects/6e68e2ea99614c628e140a6c59646315/users/208fefff0fed4fda80c40b1240b16c8f/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:28.988686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:28.988979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:28.989257 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:29.016489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] MFA Rules not processed for user `208fefff0fed4fda80c40b1240b16c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:29.017706 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:29.088189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1345/2696] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:29.110837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:29.112722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:29.178773 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48615aca-9f3e-475f-85f3-505320df76d9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:29.182852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:29.183534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:29.183822 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:29.184098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:29.192889 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:29.194340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:29.203259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1352/2697] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:39.089154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:39.199536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:46.356490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:46.357921 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:46.433357 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb04a90-819e-4420-a8df-e3696ae20bc5 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:46.438947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:46.439645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:46.439931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:46.440207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:46.442018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:46.454633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:46.456323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084acef-dc77-4778-8091-8bedb05cf4ea service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:46.468028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1346/2698] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:49.871730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:49.873212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:49.939670 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee441fa5-427a-43a1-ae84-81cd96ecb4b8 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:49.944334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:49.945257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:49.945524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:49.945804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] PATH_INFO: `/v3/users/829d4df5b22e4b3e8da932cb25a40efd` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:49.947006 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=829d4df5b22e4b3e8da932cb25a40efd)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:49.949465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.004133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829d4df5b22e4b3e8da932cb25a40efd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.011389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829d4df5b22e4b3e8da932cb25a40efd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.012091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829d4df5b22e4b3e8da932cb25a40efd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.019333 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829d4df5b22e4b3e8da932cb25a40efd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.019990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829d4df5b22e4b3e8da932cb25a40efd'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.036865 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1353/2699] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:49 2022] DELETE /identity/v3/users/829d4df5b22e4b3e8da932cb25a40efd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:26:50.042077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.042570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.042682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.060426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:50.061621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:50.120082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1347/2700] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:50.129839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.131267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:50.189123 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319d54c-ee43-40f7-8697-513cc6082577 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.192737 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.193378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.193659 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.193934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.202956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.204429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c76be-473a-4ce8-8c68-a225da20384f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.214286 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1354/2701] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:50.485418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.486837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:50.498994 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd60947-6d4d-4679-87b4-fffd746f433a None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.503644 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.504562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.504906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.505246 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] PATH_INFO: `/v3/projects/8fe938eb52d1431fb8163fba3a49f8f6` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.506480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe938eb52d1431fb8163fba3a49f8f6)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.508997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.554198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe938eb52d1431fb8163fba3a49f8f6', 'request_id': 'req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.560119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe938eb52d1431fb8163fba3a49f8f6', 'request_id': 'req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.566969 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe938eb52d1431fb8163fba3a49f8f6', 'request_id': 'req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.577447 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] The token cache is being invalidate because project 8fe938eb52d1431fb8163fba3a49f8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:50.577729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.578352 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-418a6b20-c97e-4f8b-b9e7-03fd0dadcc41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:50.579883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1348/2702] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:50 2022] DELETE /identity/v3/projects/8fe938eb52d1431fb8163fba3a49f8f6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:50.585910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.586200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.586481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.603500 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:50.604683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:50.621363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:50.667063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1355/2703] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:50.675429 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.675721 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.676002 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.693917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:50.695831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:50.703837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1349/2704] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:50.707657 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.717890 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ad1dc-8128-4faf-bfb2-b01e43bff4be None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.723590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.724271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.724563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.724844 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.725585 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:50.725939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.727662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.742875 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1356/2705] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:26:50.744084 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.748664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:50.754124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1741ae-cc4a-4b1c-a966-2fd207f27799 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.760747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.762083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.762525 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.763008 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.763795 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.766053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.793454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1350/2706] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:50 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:50.797538 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.808394 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec40d5-ed1e-4fd7-9d62-6e21e61faf8c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.872170 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.872855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.873142 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.873420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.874115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.876128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.898072 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1357/2707] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:50 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:50.902423 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.913902 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a495f3-0779-4ab8-805a-c75a0d857378 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.922985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.923774 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.924125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.924466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.925320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.927036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.938308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1351/2708] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:50.942297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.953355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7cdbf-977e-4862-ac29-52078f3116e7 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.957959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.958612 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.958900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.959184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:50.959935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:50.961577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcffa6e-ce95-4997-bb20-35303dc31886 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:50.971275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1358/2709] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:26:50.975201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:50.984993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397fcf6-a4da-4c54-92c6-ffb468808c4d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:50.989638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:50.990513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:50.990776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:50.991053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] PATH_INFO: `/v3/projects/4d21a6595248431ea6480acec2ac2903/users/31a2e48aca3346eda496fa64546bc42b/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:51.006610 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d21a6595248431ea6480acec2ac2903, user_id=31a2e48aca3346eda496fa64546bc42b, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:51.010379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:51.027199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1352/2710] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:50 2022] PUT /identity/v3/projects/4d21a6595248431ea6480acec2ac2903/users/31a2e48aca3346eda496fa64546bc42b/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:51.038125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:51.038831 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:51.039110 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:51.073592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] MFA Rules not processed for user `31a2e48aca3346eda496fa64546bc42b`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:51.074848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:51.139556 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1359/2711] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:51.151233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:51.152954 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:51.208488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3aed9-8972-4f19-bf6b-03a0fd9754d0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:51.212517 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:51.213149 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:51.213431 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:51.213704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:51.221878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:51.223330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079f0cd-5f6a-4c35-832d-696764448caa service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:51.232638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1353/2712] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:52.049921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:52.051379 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.071542 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:52.117347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b448c3-239a-44b1-9fa7-279e3d92ce66 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:52.122020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:52.122914 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.123197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.123508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] PATH_INFO: `/v3/users/208fefff0fed4fda80c40b1240b16c8f` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.124679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=208fefff0fed4fda80c40b1240b16c8f)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:52.126614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:52.173014 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208fefff0fed4fda80c40b1240b16c8f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.179888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208fefff0fed4fda80c40b1240b16c8f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.180535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208fefff0fed4fda80c40b1240b16c8f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.186983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208fefff0fed4fda80c40b1240b16c8f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.187619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208fefff0fed4fda80c40b1240b16c8f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.204591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1360/2713] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:52 2022] DELETE /identity/v3/users/208fefff0fed4fda80c40b1240b16c8f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:52.209184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.209773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.209773 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.227506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:52.228694 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.286797 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1354/2714] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:52.296668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:52.298046 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.357362 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6ada6-96eb-4128-9180-a1b1d9f36303 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:52.360923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:52.361590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.361871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.362169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.372271 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:52.373708 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47c3c2-5fa7-4ea3-9b3a-78080c3cea4e service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:52.383779 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1361/2715] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:52.645803 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:52.647288 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.664554 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36478673-dfca-4f77-a58b-5dbc340c7d42 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:52.670752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:52.671824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.672082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.672404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] PATH_INFO: `/v3/projects/6e68e2ea99614c628e140a6c59646315` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.673611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e68e2ea99614c628e140a6c59646315)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:52.675663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:52.730240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:52.737270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e68e2ea99614c628e140a6c59646315', 'request_id': 'req-1ef70eba-5fad-41ad-a68e-3e8179d14bde'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.745416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e68e2ea99614c628e140a6c59646315', 'request_id': 'req-1ef70eba-5fad-41ad-a68e-3e8179d14bde'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.753765 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e68e2ea99614c628e140a6c59646315', 'request_id': 'req-1ef70eba-5fad-41ad-a68e-3e8179d14bde'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.765820 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] The token cache is being invalidate because project 6e68e2ea99614c628e140a6c59646315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:52.766200 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.767134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ef70eba-5fad-41ad-a68e-3e8179d14bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:52.768992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1355/2716] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:52 2022] DELETE /identity/v3/projects/6e68e2ea99614c628e140a6c59646315 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:52.774327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.774676 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.774885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.794879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:52.796118 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.832057 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:52.869850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1362/2717] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:52.877414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.877907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.878198 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.899308 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:52.900533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:52.908479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1356/2718] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:52.912986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:52.922863 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4b3a-c3a6-4349-bb46-9a0b49a67dce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:52.927449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:52.928102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.928387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.928664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.929478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:26:52.929784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:52.931416 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:52.942848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1363/2719] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:26:52.947766 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:52.958279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a893d3e-7a35-465a-8d7c-39dbfe117a0c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:52.963718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:52.964373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:52.964653 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:52.964926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:52.965613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:52.967553 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:52.993170 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1357/2720] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:52 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:52.997203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:53.007868 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671ab07-d594-4354-b04f-436c0e3c7460 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:53.064257 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:53.064905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.065181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.065452 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.066137 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:53.068070 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:53.089400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1364/2721] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:52 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:53.093515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:53.104537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ec23-a4b0-4499-b2f0-b0115a466e69 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:53.109328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:53.109987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.110268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.110570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.111320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:53.112957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:53.124116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1358/2722] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:53.127938 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:53.138436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0580-3894-48e0-8a3b-f0545d706b69 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:53.143637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:53.144289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.144572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.144846 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.145565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:53.147140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fde6f-dd69-4193-8a27-f9c0ffe3ae79 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:53.157243 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1365/2723] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:53.161181 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:53.170835 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174ad7f-4159-4c02-9ddf-6fa5a22f8675 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:53.175333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:53.176190 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.176467 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.176737 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] PATH_INFO: `/v3/projects/5513602e070b4ea09fa3b12a2b259626/users/12845eb9ee8642b295ad4dcbe125bfd2/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.191701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5513602e070b4ea09fa3b12a2b259626, user_id=12845eb9ee8642b295ad4dcbe125bfd2, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:53.194386 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:53.207421 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1359/2724] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:53 2022] PUT /identity/v3/projects/5513602e070b4ea09fa3b12a2b259626/users/12845eb9ee8642b295ad4dcbe125bfd2/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:53.212964 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055085a2-1fa4-4319-98d0-8b8acd691a91 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.213258 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055085a2-1fa4-4319-98d0-8b8acd691a91 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.213535 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055085a2-1fa4-4319-98d0-8b8acd691a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.231372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-055085a2-1fa4-4319-98d0-8b8acd691a91 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:53.232565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-055085a2-1fa4-4319-98d0-8b8acd691a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:53.295392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1366/2725] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:26:53.305699 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:53.307982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:53.365174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945d1b1-efcc-40c4-ab83-061ccc13ebc7 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:53.369404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:53.370053 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:53.370333 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:53.370607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:53.380605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:53.382933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:53.393408 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1360/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:57.471879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.472361 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.472482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.503668 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] MFA Rules not processed for user `12845eb9ee8642b295ad4dcbe125bfd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:57.504935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:57.558253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:57.597956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1367/2727] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:57.603139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.613128 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2c05b-8cc2-4334-af6b-e0d0f0a2cdac service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.619658 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.620326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.620605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.620876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.621572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:57.623512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:57.654839 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1361/2728] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:26:57 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:26:57.655888 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.667087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3e4a6-3760-419f-a8f9-b6f9a21688fa None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.734040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.734845 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.735115 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.735421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.736135 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:57.738293 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:57.763169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1368/2729] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:26:57 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:26:57.767926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.781690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0a8f-cdb4-4c23-bcea-6fad340b0021 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.798125 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.798826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.799109 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.799442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.800719 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:57.806672 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:57.819085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1362/2730] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:57.823543 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.839872 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2006-1171-4c68-8ba1-f05f39e6a79c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.844900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.845886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.846175 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.846486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] PATH_INFO: `/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844/users/85bb9c94e1914dd7988d36d00ad9116a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.863393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc374d8a7cc47b3bb49b71fdb07b844, user_id=85bb9c94e1914dd7988d36d00ad9116a, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:57.866063 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:57.880317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1369/2731] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:57 2022] PUT /identity/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844/users/85bb9c94e1914dd7988d36d00ad9116a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:57.885336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.896786 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99428efa-c193-4aa9-8901-514d86462cd8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.935836 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:57.962906 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.963633 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.963917 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:57.964192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:57.964922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:57.966552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:57.977676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1363/2732] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:26:57.981939 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:57.993582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4d5f-0aaa-4201-aecb-60a201086cea admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:57.998304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:57.999209 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:57.999710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:58.000023 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] PATH_INFO: `/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844/users/85bb9c94e1914dd7988d36d00ad9116a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:58.002030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc374d8a7cc47b3bb49b71fdb07b844, user_id=85bb9c94e1914dd7988d36d00ad9116a, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:58.004743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7fc9b-2384-4ac3-81aa-e7cd6527aa24 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:58.016282 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1370/2733] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:26:57 2022] PUT /identity/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844/users/85bb9c94e1914dd7988d36d00ad9116a/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:58.446838 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:58.448618 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:58.463337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4d19-13bc-4901-9191-fdc18030cc12 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:58.527833 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:58.528718 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:58.529003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:58.529273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] PATH_INFO: `/v3/users/31a2e48aca3346eda496fa64546bc42b` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:58.530486 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a2e48aca3346eda496fa64546bc42b)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:58.532550 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:58.583878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a2e48aca3346eda496fa64546bc42b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:58.584637 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a2e48aca3346eda496fa64546bc42b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:58.593207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a2e48aca3346eda496fa64546bc42b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:58.593934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a2e48aca3346eda496fa64546bc42b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:58.608026 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a2e48aca3346eda496fa64546bc42b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:58.626103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1364/2734] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:26:58 2022] DELETE /identity/v3/users/31a2e48aca3346eda496fa64546bc42b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:26:58.628215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:58.628687 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:58.629075 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:58.648502 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:26:58.649900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:58.720501 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1371/2735] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:26:58.732095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:58.735770 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:58.816457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423379f-e433-4a10-a760-f93791d7c233 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:58.822107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:58.822761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:58.823430 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:58.823804 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:58.833065 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:58.836776 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:58.838975 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e680d77-224c-48e6-83a4-97f9c36c35a0 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:58.859559 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1365/2736] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:26:59.598569 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:26:59.600044 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:26:59.612975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd9c6cc-d954-443e-b714-924ba4f5569a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:26:59.621510 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:26:59.622815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:26:59.623102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:26:59.623418 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] PATH_INFO: `/v3/projects/4d21a6595248431ea6480acec2ac2903` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:26:59.624639 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d21a6595248431ea6480acec2ac2903)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:26:59.626618 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:26:59.656490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:26:59.686180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d21a6595248431ea6480acec2ac2903', 'request_id': 'req-cb66889c-b6f3-445c-b6c3-9938023848a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:59.693318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d21a6595248431ea6480acec2ac2903', 'request_id': 'req-cb66889c-b6f3-445c-b6c3-9938023848a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:59.700908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d21a6595248431ea6480acec2ac2903', 'request_id': 'req-cb66889c-b6f3-445c-b6c3-9938023848a6'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:59.716222 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] The token cache is being invalidate because project 4d21a6595248431ea6480acec2ac2903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:26:59.716604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:59.717662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:26:59.719388 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1372/2737] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:26:59 2022] DELETE /identity/v3/projects/4d21a6595248431ea6480acec2ac2903 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:27:03.179895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a898dc-0761-4c42-94da-45fe86bcc5da None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:03.180736 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a898dc-0761-4c42-94da-45fe86bcc5da None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:03.181014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a898dc-0761-4c42-94da-45fe86bcc5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:03.209163 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-31a898dc-0761-4c42-94da-45fe86bcc5da None None] MFA Rules not processed for user `85bb9c94e1914dd7988d36d00ad9116a`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:27:03.211027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-31a898dc-0761-4c42-94da-45fe86bcc5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:03.291747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1366/2738] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:27:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:27:03.308517 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:27:03.312011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:03.400068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66889c-b6f3-445c-b6c3-9938023848a6 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:27:03.405584 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:27:03.406241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:03.406530 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:03.406812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:03.408636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:03.467481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:27:03.469086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:27:03.478881 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1373/2739] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:27:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:27:04.417030 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:04.417511 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:04.417607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:04.435996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] MFA Rules not processed for user `12845eb9ee8642b295ad4dcbe125bfd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:27:04.437456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:04.444501 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1367/2740] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:27:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:27:12.911085 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:27:12.912498 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:12.962927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:27:12.984765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff739f-72ea-4f23-8416-b75d9834cb70 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:27:12.989754 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:27:12.990403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:12.990686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:12.991002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:12.992767 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:13.005306 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:27:13.006907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440af95-98b5-45df-b249-b89f65fb00aa service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:27:13.017950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1374/2741] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:27:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:27:14.430337 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=39 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:27:20.070007 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:27:20.071480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:20.126224 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e541388-9bf4-4e87-87c3-5a2b2fc8ab67 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:27:20.129747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:27:20.130378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:20.130656 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:20.130924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:20.132669 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:20.142419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:27:20.143886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:27:20.151811 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1368/2742] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:27:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:27:23.012599 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=10 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:27:30.148318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=26 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:27:34.257079 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a52-4613-4b1a-977c-8c9cec447125 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:34.257620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a52-4613-4b1a-977c-8c9cec447125 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:34.257620 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a52-4613-4b1a-977c-8c9cec447125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:34.274937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a159a52-4613-4b1a-977c-8c9cec447125 None None] MFA Rules not processed for user `9e41cedaabc54e408cdddee2e5cb140d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:27:34.276229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a159a52-4613-4b1a-977c-8c9cec447125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:27:34.330193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1375/2743] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:27:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:27:37.761044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:27:37.773890 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9d67-b1dd-44ca-b82f-72d06606693a service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:27:37.778410 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:27:37.779108 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:27:37.779677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:27:37.780152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:27:37.790841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:27:37.792612 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:27:37.804745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1369/2744] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:27:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:27:44.323490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=25 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:27:47.798988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:28:37.302383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:28:37.302918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:28:37.302918 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:28:37.321764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] MFA Rules not processed for user `9e41cedaabc54e408cdddee2e5cb140d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:28:37.322957 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:28:37.329824 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1376/2745] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:28:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:28:47.315613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:21.364894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:21.366287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:21.431500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc66bc1-cee6-4541-b365-ce989d54a0ce service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:21.434965 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:21.435627 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:21.435909 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:21.436179 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:21.437918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:21.451621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:21.452947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:21.465837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1370/2746] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:28.249742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:28.259534 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e169c-a3f8-4845-8dda-acdeaa3d021f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:28.263698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:28.265266 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:28.265577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:28.265852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:28.267976 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:28.282086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:28.283539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:28.294956 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1377/2747] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:31.460055 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=26 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:38.289555 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:40.541392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:40.542948 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:40.558907 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2f9d-4e0f-4da1-8c4f-694ac99fa513 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:40.564061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:40.564708 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:40.564986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:40.565264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:40.567227 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:40.582054 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:40.583980 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:40.600655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1371/2748] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:40.651174 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:40.652797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:40.732008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb16692-5f9c-4641-8de4-0655b403d199 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:40.739200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:40.740741 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:40.741020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:40.741352 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] PATH_INFO: `/v3/users/9e41cedaabc54e408cdddee2e5cb140d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:40.743233 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e41cedaabc54e408cdddee2e5cb140d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:40.745922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:40.793158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e41cedaabc54e408cdddee2e5cb140d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:40.799800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e41cedaabc54e408cdddee2e5cb140d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:40.800540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e41cedaabc54e408cdddee2e5cb140d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:40.807760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e41cedaabc54e408cdddee2e5cb140d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:40.808387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-da53d823-30c3-45db-96de-da174ef892c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e41cedaabc54e408cdddee2e5cb140d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:40.825580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1378/2749] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:40 2022] DELETE /identity/v3/users/9e41cedaabc54e408cdddee2e5cb140d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:29:40.994494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.005201 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3473fbd-1635-49cf-bc97-861aa43a52c2 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.062988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.063815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.064102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.064379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] PATH_INFO: `/v3/projects/3676f20a4e0d4ec8be71d5aaab607aa7` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.065555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3676f20a4e0d4ec8be71d5aaab607aa7)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.067491 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.111534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3676f20a4e0d4ec8be71d5aaab607aa7', 'request_id': 'req-3d48f9c0-051a-4422-8bbd-e41706e7f452'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:41.118085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3676f20a4e0d4ec8be71d5aaab607aa7', 'request_id': 'req-3d48f9c0-051a-4422-8bbd-e41706e7f452'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:41.125079 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3676f20a4e0d4ec8be71d5aaab607aa7', 'request_id': 'req-3d48f9c0-051a-4422-8bbd-e41706e7f452'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:41.136045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] The token cache is being invalidate because project 3676f20a4e0d4ec8be71d5aaab607aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:41.136334 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:41.136956 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d48f9c0-051a-4422-8bbd-e41706e7f452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:41.138468 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1372/2750] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:40 2022] DELETE /identity/v3/projects/3676f20a4e0d4ec8be71d5aaab607aa7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:41.143652 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.143941 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.144218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.163681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:41.165493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:41.228674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1379/2751] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:41.236391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.236680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.236953 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.257269 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:41.259642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:41.268515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1373/2752] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:41.272697 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.282740 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cf1c9-c718-4b5f-baa2-55a1d5fc2aa6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.287984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.288653 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.288944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.289253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.289987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:29:41.290278 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.291910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.303727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1380/2753] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:29:41.308657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.317733 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0830b-ff9e-4fcb-a713-6dfc5f4e4877 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.322947 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.323613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.323889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.324159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.324831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.326761 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.351097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1374/2754] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/projects => generated 403 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:41.355649 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.365735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbd6b5-6548-4150-919f-3df435bce184 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.370766 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:41.425031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.425698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.425985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.426264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.426961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.428908 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.449215 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1381/2755] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/users => generated 302 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:41.453370 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.467191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708619c1-b5dc-483b-a8bf-0ebb9a579475 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.471971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.472648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.472933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.473211 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.473937 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.475589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.486675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1375/2756] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:41.490617 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.501456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e06160-eba4-4cca-b9b9-625b08bd6a74 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.506042 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.506695 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.506978 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.507275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.508058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.509577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62172bf-52fd-4635-8070-cafc16f22154 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.519886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1382/2757] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:41.523756 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.533671 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61881e-1ca9-45f1-a2d6-a1b8e40bcbe0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.538363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.539785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.540037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.540313 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] PATH_INFO: `/v3/projects/5bae47dff1a446eba87f87c45d21259e/users/4302dae884e54398a8185b5b36d77b73/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.554675 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bae47dff1a446eba87f87c45d21259e, user_id=4302dae884e54398a8185b5b36d77b73, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.558010 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.570605 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1376/2758] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:41 2022] PUT /identity/v3/projects/5bae47dff1a446eba87f87c45d21259e/users/4302dae884e54398a8185b5b36d77b73/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:41.580873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.581166 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.581446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.608476 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] MFA Rules not processed for user `4302dae884e54398a8185b5b36d77b73`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:41.609712 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:41.674017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1383/2759] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:29:41.678505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.692452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:41.743024 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8f04-3627-4666-a9fd-8c1f7625840d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.747741 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.748395 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.748676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.748949 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.749629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.751585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.780270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1377/2760] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:41.782316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.794213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53debd7f-10b0-4b08-808c-7ae7a019e3ce None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.856010 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.856602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.856883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.857155 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.857860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.859812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.881380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1384/2761] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:41 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:41.885732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.896982 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c986ef-9c50-4d71-91e2-d6e4e7a4fc13 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.902680 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.903393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.903678 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.903950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.904691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.906321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.921403 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1378/2762] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:41.922706 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.929337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:41.934423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b98d-0332-42ed-90a0-a1b91f266c8c admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:41.940689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:41.941590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:41.941871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:41.942144 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] PATH_INFO: `/v3/projects/2646646b4f3346be974d788b55ca5a01/users/e3ee3617a8574055b752c6f23dd99e06/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:41.958307 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2646646b4f3346be974d788b55ca5a01, user_id=e3ee3617a8574055b752c6f23dd99e06, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:41.961577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:41.978949 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1385/2763] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:41 2022] PUT /identity/v3/projects/2646646b4f3346be974d788b55ca5a01/users/e3ee3617a8574055b752c6f23dd99e06/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:41.983147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:41.993987 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f9797-7c1d-4b1c-8f5e-1a3e5d0a65df admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:42.054082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:42.054852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.055151 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.055515 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.056258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:42.057908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04cee3-f909-41c0-a3fb-01892d1350c1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:42.072349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1379/2764] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:42.073637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:42.083795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75bbb4-5250-4c90-804e-67396a7ca150 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:42.088800 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:42.089682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.089962 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.090232 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] PATH_INFO: `/v3/projects/2646646b4f3346be974d788b55ca5a01/users/e3ee3617a8574055b752c6f23dd99e06/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.092248 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2646646b4f3346be974d788b55ca5a01, user_id=e3ee3617a8574055b752c6f23dd99e06, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:42.094867 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:42.106487 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1386/2765] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:42 2022] PUT /identity/v3/projects/2646646b4f3346be974d788b55ca5a01/users/e3ee3617a8574055b752c6f23dd99e06/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:42.117805 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.118096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.118374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.145533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] MFA Rules not processed for user `e3ee3617a8574055b752c6f23dd99e06`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:42.146744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:42.213784 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1380/2766] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:42.224656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:42.226408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:42.288091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fd1ed-059f-4937-b117-9ce8d7885f29 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:42.291995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:42.292640 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.292925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.293203 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.302186 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:42.303662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4443c5e-ebe4-4df7-a890-64236a1b920a service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:42.313322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1387/2767] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:42.442412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:42.453157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:42.508891 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc1aa4-20ba-4460-a6fe-fb3469b617c6 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:42.515443 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:42.516309 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.516590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.516865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] PATH_INFO: `/v3/users/4302dae884e54398a8185b5b36d77b73` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.518319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4302dae884e54398a8185b5b36d77b73)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:42.520302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:42.565601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4302dae884e54398a8185b5b36d77b73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:42.566250 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4302dae884e54398a8185b5b36d77b73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:42.573472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4302dae884e54398a8185b5b36d77b73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:42.574097 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4302dae884e54398a8185b5b36d77b73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:42.585628 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4302dae884e54398a8185b5b36d77b73'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:42.598245 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1381/2768] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:42 2022] DELETE /identity/v3/users/4302dae884e54398a8185b5b36d77b73 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:42.602722 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.603008 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.603339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.621077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:42.622197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:42.681165 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1388/2769] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:42.690607 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:42.691993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:42.747943 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9c11b-ab24-4dcc-a1a7-e27160142b57 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:42.751533 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:42.752164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:42.752439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:42.752710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:42.762345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:42.763798 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:42.773241 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1382/2770] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:43.035771 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.037202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:43.050565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a475610-6b76-4675-969c-1e33317631e6 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:43.056098 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:43.056780 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.057069 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.057347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] PATH_INFO: `/v3/projects/5bae47dff1a446eba87f87c45d21259e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.058546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bae47dff1a446eba87f87c45d21259e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:43.060490 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:43.107423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bae47dff1a446eba87f87c45d21259e', 'request_id': 'req-9d633872-7153-445c-baa3-0c69880a0043'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.112866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bae47dff1a446eba87f87c45d21259e', 'request_id': 'req-9d633872-7153-445c-baa3-0c69880a0043'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.119703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bae47dff1a446eba87f87c45d21259e', 'request_id': 'req-9d633872-7153-445c-baa3-0c69880a0043'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.122192 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:43.132990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] The token cache is being invalidate because project 5bae47dff1a446eba87f87c45d21259e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:43.133308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.133899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.135454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1389/2771] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:43 2022] DELETE /identity/v3/projects/5bae47dff1a446eba87f87c45d21259e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:29:43.139438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.149122 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d5e6a-849c-40db-ad3c-5ceff147c86a service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:43.178878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:43.209604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:43.210481 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.210764 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.211585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] PATH_INFO: `/v3/users/e3ee3617a8574055b752c6f23dd99e06` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.212780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ee3617a8574055b752c6f23dd99e06)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:43.214723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:43.269331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ee3617a8574055b752c6f23dd99e06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.270045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ee3617a8574055b752c6f23dd99e06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.279077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ee3617a8574055b752c6f23dd99e06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.280184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ee3617a8574055b752c6f23dd99e06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.293089 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-4e2ae4d4-e9b6-484f-8f44-8ff51dd7e71f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ee3617a8574055b752c6f23dd99e06'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.306584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1383/2772] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:43 2022] DELETE /identity/v3/users/e3ee3617a8574055b752c6f23dd99e06 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:43.579264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.580698 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:43.596150 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633872-7153-445c-baa3-0c69880a0043 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:43.655738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:43.656627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.656912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.657189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] PATH_INFO: `/v3/projects/2646646b4f3346be974d788b55ca5a01` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.658466 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2646646b4f3346be974d788b55ca5a01)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:43.660412 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:43.714720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2646646b4f3346be974d788b55ca5a01', 'request_id': 'req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.720672 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2646646b4f3346be974d788b55ca5a01', 'request_id': 'req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.727370 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2646646b4f3346be974d788b55ca5a01', 'request_id': 'req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.741573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] The token cache is being invalidate because project 2646646b4f3346be974d788b55ca5a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:43.741871 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.742546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-a4c7864e-0d2f-49f3-a807-da8095f1ca8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:43.744218 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1390/2773] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:43 2022] DELETE /identity/v3/projects/2646646b4f3346be974d788b55ca5a01 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:43.750439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.751028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.751028 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.770188 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:43.771470 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:43.833439 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1384/2774] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:43.841658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.841951 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.842229 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.860563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:43.861789 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:43.869738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1391/2775] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:29:43.873885 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.884782 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4d314-27ef-4a4a-82a1-75c1cf2489ea None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:43.889271 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:43.889908 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.890184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.890454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.891168 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80416) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:29:43.891512 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:43.893355 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:43.905785 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1385/2776] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:29:43.910154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.919191 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ecede-397e-47c1-b8b8-84183ff2073a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:43.923743 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:43.924380 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:43.924660 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:43.924932 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:43.925616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:43.927536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:43.952409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1392/2777] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:43 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:43.956404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:43.966952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9dad4-95c5-4636-acbd-8614fef80aa8 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.000124 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:44.025571 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.026229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.026508 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.026780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.027497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.029399 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.050193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1386/2778] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:43 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:44.054270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.065262 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2f08a-1b60-4b5c-b02e-6fdbb068c734 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.070915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.072287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.072287 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.072491 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.073083 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.074735 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.085415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1393/2779] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:44.089589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.099534 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801f205-37fb-4f36-b60e-095eb2ddb615 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.104193 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.104840 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.105120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.105393 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.106133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.107746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8833-6b95-4ec2-b887-a69a505da744 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.118080 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1387/2780] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:44.122210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.132128 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e182457-0daa-4545-b896-83e0bb760bc5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.137425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.138295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.138577 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.138852 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] PATH_INFO: `/v3/projects/250d46eb8b464edb978bf3cd4f8dfab1/users/a5ac7b01aa5844ac80ab2b2e366191e2/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.153564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250d46eb8b464edb978bf3cd4f8dfab1, user_id=a5ac7b01aa5844ac80ab2b2e366191e2, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.156304 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.169372 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1394/2781] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:44 2022] PUT /identity/v3/projects/250d46eb8b464edb978bf3cd4f8dfab1/users/a5ac7b01aa5844ac80ab2b2e366191e2/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:44.179364 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.180157 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.180482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.206887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] MFA Rules not processed for user `a5ac7b01aa5844ac80ab2b2e366191e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:44.208120 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:44.270977 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1388/2782] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:44.275565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.296339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:44.340926 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492997-e38f-43af-b51b-72b0c9518027 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.345813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.346479 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.346760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.347035 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.347769 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.349674 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.380019 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1395/2783] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:44 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:44.383848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.395837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc937-21e7-415d-91cb-7037288b7523 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.465523 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.466287 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.466565 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.466837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] PATH_INFO: `/v3/users` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.467556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.469482 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.491701 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1389/2784] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:44 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:29:44.496087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.506658 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696aba-c9db-4a64-93a3-5a7f3e6fbca4 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.513031 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.513885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.514391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.514883 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.515990 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.517870 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.533041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1396/2785] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:44.536946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.547753 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d4e01-c4a4-4d44-a778-537abe3aee5f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.552438 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.553339 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.553621 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.553904 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] PATH_INFO: `/v3/projects/b66e5ecee3714acb8c187c11ecf790df/users/d3962307f7a448f0b68d3c4c46aa362d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.569239 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66e5ecee3714acb8c187c11ecf790df, user_id=d3962307f7a448f0b68d3c4c46aa362d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.572116 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.584688 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1390/2786] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:44 2022] PUT /identity/v3/projects/b66e5ecee3714acb8c187c11ecf790df/users/d3962307f7a448f0b68d3c4c46aa362d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:44.588562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.598421 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7f54-b71f-4a33-8626-bab66dddae89 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.654179 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.654839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.655125 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.655432 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.656153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.657742 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96849b-322f-4bc6-88d6-0d739f3ea87b admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.667917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1397/2787] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:44.672154 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.684458 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4d223-2730-40e3-afdd-7b8f2f46c711 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.689147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.690020 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.690298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.690578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] PATH_INFO: `/v3/projects/b66e5ecee3714acb8c187c11ecf790df/users/d3962307f7a448f0b68d3c4c46aa362d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.692588 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66e5ecee3714acb8c187c11ecf790df, user_id=d3962307f7a448f0b68d3c4c46aa362d, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.695212 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.701136 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:44.707106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1391/2788] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:44 2022] PUT /identity/v3/projects/b66e5ecee3714acb8c187c11ecf790df/users/d3962307f7a448f0b68d3c4c46aa362d/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:44.719429 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.719904 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.720001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.746685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] MFA Rules not processed for user `d3962307f7a448f0b68d3c4c46aa362d`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:44.747917 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:44.824478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1398/2789] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:44.835584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:44.838150 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:44.898162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fe501-be1b-4545-a06f-15d743fbd41d admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:44.902746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:44.903442 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:44.903728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:44.904113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:44.914132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:44.916312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:44.926877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1392/2790] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:45.015796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:45.026813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce561-6417-46c5-ae13-ec60a65c903c None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:45.030497 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:45.031159 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:45.031475 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:45.031751 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:45.045074 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:45.095906 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:45.097365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:45.107988 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1399/2791] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:49.470476 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:49.472073 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:49.533875 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb14473-bd71-4657-b77f-4b622e12bde2 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:49.537519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:49.538162 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:49.538446 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:49.538724 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:49.541243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:49.553528 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:49.554985 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4459c-86e2-4faf-92b9-8aebdf54621f service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:49.563081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1393/2792] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:49.832403 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:49.895252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53d645-b43c-40ac-b1d0-eea6d9f81dbd service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:49.899685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:49.900641 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:49.900928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:49.901202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] PATH_INFO: `/v3/users/a5ac7b01aa5844ac80ab2b2e366191e2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:49.902427 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ac7b01aa5844ac80ab2b2e366191e2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:49.904615 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:49.952245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ac7b01aa5844ac80ab2b2e366191e2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:49.959724 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ac7b01aa5844ac80ab2b2e366191e2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:49.960419 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ac7b01aa5844ac80ab2b2e366191e2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:49.968899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ac7b01aa5844ac80ab2b2e366191e2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:49.969580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ac7b01aa5844ac80ab2b2e366191e2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:49.987923 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1400/2793] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:49 2022] DELETE /identity/v3/users/a5ac7b01aa5844ac80ab2b2e366191e2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:49.992380 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:49.992674 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:49.992952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:50.011418 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:50.012642 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:50.072018 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1394/2794] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:50.081573 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:50.083030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:50.122681 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:50.146690 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2153274-ecd0-4e36-8eaa-661ab44db55d admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:50.150591 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:50.151253 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:50.151545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:50.151809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:50.160716 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:50.162138 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:50.171415 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1401/2795] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:50.446568 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:50.448105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:50.460256 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3709a-a1c9-40a6-a8a7-12c4e71edee4 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:50.465140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:50.466085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:50.466366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:50.466638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] PATH_INFO: `/v3/projects/250d46eb8b464edb978bf3cd4f8dfab1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:50.468762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=250d46eb8b464edb978bf3cd4f8dfab1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:50.470686 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:50.512560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:50.520584 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250d46eb8b464edb978bf3cd4f8dfab1', 'request_id': 'req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.526848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250d46eb8b464edb978bf3cd4f8dfab1', 'request_id': 'req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.534126 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250d46eb8b464edb978bf3cd4f8dfab1', 'request_id': 'req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.546061 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] The token cache is being invalidate because project 250d46eb8b464edb978bf3cd4f8dfab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:50.546564 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.547406 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.549323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1395/2796] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:50 2022] DELETE /identity/v3/projects/250d46eb8b464edb978bf3cd4f8dfab1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:50.553317 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:50.563866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17762d-66e2-45a1-a6d6-c34be88926a3 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:50.627461 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:50.628339 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:50.628619 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:50.628892 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] PATH_INFO: `/v3/users/d3962307f7a448f0b68d3c4c46aa362d` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:50.630094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3962307f7a448f0b68d3c4c46aa362d)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:50.632050 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:50.688355 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3962307f7a448f0b68d3c4c46aa362d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.695378 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3962307f7a448f0b68d3c4c46aa362d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.695973 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3962307f7a448f0b68d3c4c46aa362d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.703997 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3962307f7a448f0b68d3c4c46aa362d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.704749 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-fd795dd9-5845-4eb4-a481-2e28c94e967a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3962307f7a448f0b68d3c4c46aa362d'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:50.722910 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1402/2797] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:50 2022] DELETE /identity/v3/users/d3962307f7a448f0b68d3c4c46aa362d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:50.996328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:50.997771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:51.012006 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd79b17-d3a0-4d2c-b9f7-8b3fe359dbaf admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.067993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.068832 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.069106 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.069372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] PATH_INFO: `/v3/projects/b66e5ecee3714acb8c187c11ecf790df` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.070519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66e5ecee3714acb8c187c11ecf790df)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.072531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.117366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66e5ecee3714acb8c187c11ecf790df', 'request_id': 'req-1ce131e6-0913-44e6-958f-a44393369864'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.123632 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66e5ecee3714acb8c187c11ecf790df', 'request_id': 'req-1ce131e6-0913-44e6-958f-a44393369864'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.130979 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66e5ecee3714acb8c187c11ecf790df', 'request_id': 'req-1ce131e6-0913-44e6-958f-a44393369864'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.142105 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] The token cache is being invalidate because project b66e5ecee3714acb8c187c11ecf790df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:51.142646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.143327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1ce131e6-0913-44e6-958f-a44393369864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.144829 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1396/2798] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:50 2022] DELETE /identity/v3/projects/b66e5ecee3714acb8c187c11ecf790df => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:51.150344 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.150856 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.150963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.169172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:51.170420 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:51.220132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:51.233456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1403/2799] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:51.241379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.241744 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.242021 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.260966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:51.262483 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:51.273062 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1397/2800] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:51.278189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.290197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22562fd6-5663-4293-a34f-351e9782d10a None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.295880 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.297134 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.297545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.297900 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.298791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:29:51.299647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.301281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.319086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1404/2801] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:29:51.324264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.333456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00aa9d-15c6-4b2e-8136-b463e346cc73 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.338730 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.339409 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.339690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.339964 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.340655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.342562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.375102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1398/2802] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:51.379094 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.389827 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3deb-8790-44c8-8caf-77a168198840 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.445347 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.446017 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.446935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.447212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.447924 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.449853 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.477127 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1405/2803] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/users => generated 281 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:51.482477 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.492783 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef185b1-101f-417b-9612-81919883c6ac admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.497882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.498531 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.498812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.499086 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.499845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.501444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.512492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1399/2804] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:51.516443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.526524 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f624c9-ccea-47c2-8257-058b0a4e1bd2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.531095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.531784 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.532064 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.532335 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.533049 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.534636 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12ecd3-28c0-4ffb-b595-22406f3f0bd6 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.544533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1406/2805] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:51.548436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.559092 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4626ae3-98b4-4135-9258-da9dc42c2ae6 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.563747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.564602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.564876 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.565144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] PATH_INFO: `/v3/projects/6ffff4e1bae64760b501bf75d1970b35/users/0b70dc80970049d5a0e2edb5a1a6db88/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.580778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffff4e1bae64760b501bf75d1970b35, user_id=0b70dc80970049d5a0e2edb5a1a6db88, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.583489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.585651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:51.598259 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1400/2806] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:51 2022] PUT /identity/v3/projects/6ffff4e1bae64760b501bf75d1970b35/users/0b70dc80970049d5a0e2edb5a1a6db88/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:51.608783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.609066 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.609340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.636651 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] MFA Rules not processed for user `0b70dc80970049d5a0e2edb5a1a6db88`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:51.637848 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:51.701967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1407/2807] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:51.733379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.735045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:51.797661 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760946d-80ea-4edc-bf87-38ace22bb926 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.801414 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.802148 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.802433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.802710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.812229 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.813676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958d68c-4c7c-4865-a757-30b79e69e307 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.823505 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1401/2808] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:51.838026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:51.888847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:51.902838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104b529-fb78-459d-83c6-3b07909405f9 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:51.907449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:51.908351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.908633 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.908907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] PATH_INFO: `/v3/users/0b70dc80970049d5a0e2edb5a1a6db88` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:51.910099 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b70dc80970049d5a0e2edb5a1a6db88)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:51.912097 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:51.958021 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b70dc80970049d5a0e2edb5a1a6db88'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.965580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b70dc80970049d5a0e2edb5a1a6db88'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.966236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b70dc80970049d5a0e2edb5a1a6db88'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.973252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b70dc80970049d5a0e2edb5a1a6db88'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.973860 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b70dc80970049d5a0e2edb5a1a6db88'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:51.991320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1408/2809] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:29:51 2022] DELETE /identity/v3/users/0b70dc80970049d5a0e2edb5a1a6db88 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:51.996788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:51.997025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:51.997327 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.014415 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:52.015663 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:52.075000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1402/2810] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:52.084453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.085834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:52.143255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56ec5-3a26-415f-a7da-024366b1ed58 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.146886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.147557 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.147839 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.148116 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.157056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.158486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e0243-9f11-49cd-bdc3-91ccdccfc068 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.168686 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1409/2811] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:52.428358 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.429924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:52.449500 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb945edf-0cdf-452f-940a-ae2359be8628 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.456758 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.457762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.458044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.458322 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] PATH_INFO: `/v3/projects/6ffff4e1bae64760b501bf75d1970b35` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.459530 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffff4e1bae64760b501bf75d1970b35)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.461457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.507997 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffff4e1bae64760b501bf75d1970b35', 'request_id': 'req-a4111a6c-77d6-4743-b9d7-55dee3c520e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:52.514555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffff4e1bae64760b501bf75d1970b35', 'request_id': 'req-a4111a6c-77d6-4743-b9d7-55dee3c520e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:52.522042 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffff4e1bae64760b501bf75d1970b35', 'request_id': 'req-a4111a6c-77d6-4743-b9d7-55dee3c520e5'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:52.533129 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] The token cache is being invalidate because project 6ffff4e1bae64760b501bf75d1970b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:29:52.533424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:52.534031 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-a4111a6c-77d6-4743-b9d7-55dee3c520e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:29:52.535595 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1403/2812] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:29:52 2022] DELETE /identity/v3/projects/6ffff4e1bae64760b501bf75d1970b35 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 03:29:52.541454 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.541963 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.542472 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.561862 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:52.563156 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:52.622088 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1410/2813] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:52.679552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.679831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.680113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.698728 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:52.699970 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:52.707822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1404/2814] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:52.712714 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.722496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d62309-5c5c-42b9-a96c-8483c3c16a3f None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.727001 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.727815 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.728265 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.728547 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.729264 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:29:52.729571 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.731161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.739552 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:52.745654 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1411/2815] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:29:52.749152 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.758506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99720d42-c0d4-42e4-b02a-e449b855e08c None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.763673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.764318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.764603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.764878 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.765586 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.767787 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.778083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:52.794639 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1405/2816] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:52.798701 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.808909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd47204-a6aa-4f2b-9d8f-75a6f27b5ad2 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.864482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.865145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.865431 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.865709 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.866387 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.868389 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.889030 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1412/2817] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:52 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:52.893502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.904189 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbe80f-9bcb-4ecd-938a-47b7628bbdfe admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.908844 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.909859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.909859 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.910076 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.910739 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.912416 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.927320 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1406/2818] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:52.928570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.939275 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaaa16-a79d-4e16-aefa-f474c08b9555 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.944320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.944985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.945297 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.945572 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.946934 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.948614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946bd98-7ce4-4470-b1ec-d24eb29cf772 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:52.958788 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1413/2819] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 03:29:52.963022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:52.972973 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ed998-a20e-4abc-bda9-0195f50bf45f admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:52.977684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:52.978562 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:52.978848 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:52.979121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] PATH_INFO: `/v3/projects/ccc320b910684210b34e26267693b9c3/users/23c209aacfa24ea1a1684b2aed2bdcf2/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:52.994746 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc320b910684210b34e26267693b9c3, user_id=23c209aacfa24ea1a1684b2aed2bdcf2, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:52.997440 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.011626 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1407/2820] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:52 2022] PUT /identity/v3/projects/ccc320b910684210b34e26267693b9c3/users/23c209aacfa24ea1a1684b2aed2bdcf2/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:53.021840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.022133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.022410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.049974 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] MFA Rules not processed for user `23c209aacfa24ea1a1684b2aed2bdcf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:53.051277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:53.116841 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1414/2821] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:53.121886 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.184539 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585304-8876-4193-8b0a-883ebe7d5b28 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.189183 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.189830 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.190111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.190383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.191099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.193022 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.218214 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1408/2822] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:53.222201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.232933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e5b1-f228-4581-8f91-1e6e10356790 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.290058 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.290733 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.291020 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.291334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.292056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.293975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.308235 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:53.315145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1415/2823] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:29:53.318871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.331918 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc557a-8ec4-43dc-8f49-620d8b78dc4b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.336583 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.337353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.337634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.337990 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.338712 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.340341 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.351134 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1409/2824] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:53.355775 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.365579 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407a6a6-d5c3-4851-b0f1-269a577c58b9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.371002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.371699 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.371985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.372259 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.372981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.374604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dfea7-9542-42e2-aaa8-3d1df8b72b0f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.385381 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1416/2825] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:53.389469 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.399604 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08da3ca-0644-43af-a7cf-0bf6978e2ca2 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.404233 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.405091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.405366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.405636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] PATH_INFO: `/v3/projects/6a7de6e1596f4c288a5a11396e4bdab1/users/31878abb1f554ca19e8cfbadfe95fbfc/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.420378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7de6e1596f4c288a5a11396e4bdab1, user_id=31878abb1f554ca19e8cfbadfe95fbfc, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.423014 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.435933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1410/2826] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:53 2022] PUT /identity/v3/projects/6a7de6e1596f4c288a5a11396e4bdab1/users/31878abb1f554ca19e8cfbadfe95fbfc/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:53.447200 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.447514 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.447794 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.473471 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] MFA Rules not processed for user `31878abb1f554ca19e8cfbadfe95fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:53.474730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:53.538570 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1417/2827] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:53.543807 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.603763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa1e06-7041-42ab-9200-74e1c2fa4145 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.608312 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.608950 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.609226 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.609497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.610177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.612207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.622240 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:53.638851 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1411/2828] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:29:53.643184 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.653836 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6989f5-0fa5-429b-a9e5-70afa0857382 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.709746 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.710407 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.710692 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.710965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.711682 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.714145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.733922 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1418/2829] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:29:53.737682 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.747893 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cc140-d15e-4cb9-9164-71e229450f11 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.752424 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.753066 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.753343 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.753611 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.754318 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.755938 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.766495 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1412/2830] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:53.770664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.781163 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeeb7f-b995-441f-a1fe-5e47a9ebb0f3 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.785665 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.786525 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.786803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.787100 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] PATH_INFO: `/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd/users/8e92238b645541a9869471b308bf64c3/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.802158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb2d1687b2d4b838c949bc2700b79dd, user_id=8e92238b645541a9869471b308bf64c3, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.804755 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.817711 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1419/2831] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:53 2022] PUT /identity/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd/users/8e92238b645541a9869471b308bf64c3/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:53.821763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.831855 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3fc8-c392-4299-b9d0-2314b34d606c admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.885883 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.886537 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.886818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.887090 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.887824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.889420 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16faba4-59be-4613-a0b0-fb3bf147a578 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.899155 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1413/2832] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:29:53.903999 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:53.914016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3f03-0fc4-4c5d-8dac-6631bf37a426 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:53.919217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:53.920114 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.920395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.920673 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] PATH_INFO: `/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd/users/8e92238b645541a9869471b308bf64c3/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.922671 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb2d1687b2d4b838c949bc2700b79dd, user_id=8e92238b645541a9869471b308bf64c3, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:53.925330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:53.935553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1420/2833] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:29:53 2022] PUT /identity/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd/users/8e92238b645541a9869471b308bf64c3/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:29:53.945407 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:53.945695 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:53.946003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:53.972057 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] MFA Rules not processed for user `8e92238b645541a9869471b308bf64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:29:53.973255 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:54.039659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1414/2834] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:29:54.050785 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:54.052560 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:54.088217 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:54.116334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a82cdd-2761-4a93-821a-523b1e970f48 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:54.120693 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:54.121342 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:54.121623 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:54.121899 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:54.184732 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:54.187546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:54.200859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1421/2835] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:55.690602 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:55.692033 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:55.704209 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d9c95-3b1b-4da1-891e-fdc55c62542f None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:55.707735 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:55.708365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:55.708648 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:55.708922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:55.765268 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:55.766779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:55.771634 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:29:55.777398 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1415/2836] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:56.304879 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:56.314169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97dc99-d043-4ce0-894f-5fc3a520741c service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:56.317809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:56.318447 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:56.318726 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:56.318998 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:56.330045 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:56.331554 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:56.341216 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1422/2837] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:29:59.403899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:29:59.405419 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:59.482298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0d7a-725d-418e-887f-a4e4b0275963 service cinder] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:29:59.487768 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:29:59.488404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:29:59.488685 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:29:59.488961 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:29:59.490722 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:29:59.502521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:29:59.504147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3fe74-51d2-4c34-a9cc-b7b696bc2f89 service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:29:59.515023 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1416/2838] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:30:00.196664 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:00.258133 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f87f3-b4c4-4d53-86f2-31cdf5f9c734 service cinder] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:00.263565 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:00.264433 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:00.264760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:00.265056 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] PATH_INFO: `/v3/users/23c209aacfa24ea1a1684b2aed2bdcf2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:00.266277 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c209aacfa24ea1a1684b2aed2bdcf2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:00.268508 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:00.312933 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c209aacfa24ea1a1684b2aed2bdcf2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.319840 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c209aacfa24ea1a1684b2aed2bdcf2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.320493 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c209aacfa24ea1a1684b2aed2bdcf2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.327812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c209aacfa24ea1a1684b2aed2bdcf2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.328434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c209aacfa24ea1a1684b2aed2bdcf2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.335356 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:00.347859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1423/2839] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:30:00 2022] DELETE /identity/v3/users/23c209aacfa24ea1a1684b2aed2bdcf2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:00.352933 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:00.353218 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:00.353498 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:00.371330 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:30:00.372335 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:00.442371 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1417/2840] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:30:00.451984 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:00.453391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:00.510545 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65b6ab-953d-40bc-b9a0-8f279f5be6e9 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:00.514197 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:00.514866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:00.515149 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:00.515458 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:00.525160 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:00.526607 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:00.536154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1424/2841] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:30:00.827788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:00.829273 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:00.844748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa3013-c15c-4369-8e86-732cd180208d None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:00.849837 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:00.850702 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:00.850989 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:00.851286 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] PATH_INFO: `/v3/projects/ccc320b910684210b34e26267693b9c3` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:00.852532 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc320b910684210b34e26267693b9c3)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:00.854560 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:00.904003 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc320b910684210b34e26267693b9c3', 'request_id': 'req-2e98d033-cf9e-4e93-a674-8b66bbd9b091'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.910931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc320b910684210b34e26267693b9c3', 'request_id': 'req-2e98d033-cf9e-4e93-a674-8b66bbd9b091'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.918460 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc320b910684210b34e26267693b9c3', 'request_id': 'req-2e98d033-cf9e-4e93-a674-8b66bbd9b091'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.928971 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] The token cache is being invalidate because project ccc320b910684210b34e26267693b9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:30:00.929262 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.929882 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:00.931991 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1418/2842] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:30:00 2022] DELETE /identity/v3/projects/ccc320b910684210b34e26267693b9c3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:00.936188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:00.947073 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4bbd7-394d-439a-8812-4a0208e0cdf0 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:01.009395 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:01.010364 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:01.010648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:01.010925 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] PATH_INFO: `/v3/users/31878abb1f554ca19e8cfbadfe95fbfc` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:01.012289 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31878abb1f554ca19e8cfbadfe95fbfc)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:01.014251 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:01.081358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31878abb1f554ca19e8cfbadfe95fbfc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.089272 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31878abb1f554ca19e8cfbadfe95fbfc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.090122 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31878abb1f554ca19e8cfbadfe95fbfc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.101040 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31878abb1f554ca19e8cfbadfe95fbfc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.101646 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31878abb1f554ca19e8cfbadfe95fbfc'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.121600 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1425/2843] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:30:00 2022] DELETE /identity/v3/users/31878abb1f554ca19e8cfbadfe95fbfc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:01.407792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:01.409323 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:01.423769 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98d033-cf9e-4e93-a674-8b66bbd9b091 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:01.429328 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:01.486816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:01.487651 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:01.487932 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:01.488319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] PATH_INFO: `/v3/projects/6a7de6e1596f4c288a5a11396e4bdab1` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:01.489490 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7de6e1596f4c288a5a11396e4bdab1)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:01.491448 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:01.536449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7de6e1596f4c288a5a11396e4bdab1', 'request_id': 'req-feef94b1-4627-49e9-822f-50a967ab9cb0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.543081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7de6e1596f4c288a5a11396e4bdab1', 'request_id': 'req-feef94b1-4627-49e9-822f-50a967ab9cb0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.549823 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7de6e1596f4c288a5a11396e4bdab1', 'request_id': 'req-feef94b1-4627-49e9-822f-50a967ab9cb0'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.560194 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] The token cache is being invalidate because project 6a7de6e1596f4c288a5a11396e4bdab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:30:01.560480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.561095 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.562593 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1419/2844] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:30:01 2022] DELETE /identity/v3/projects/6a7de6e1596f4c288a5a11396e4bdab1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:01.566553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:01.567975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:01.581295 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e72511-8659-443f-9d38-a98a43e3894f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:01.589316 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:01.639985 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:01.640847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:01.641124 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:01.641391 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] PATH_INFO: `/v3/users/8e92238b645541a9869471b308bf64c3` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:01.642798 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e92238b645541a9869471b308bf64c3)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:01.644747 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:01.690494 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e92238b645541a9869471b308bf64c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.697446 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e92238b645541a9869471b308bf64c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.698154 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e92238b645541a9869471b308bf64c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.705325 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e92238b645541a9869471b308bf64c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.705986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-2e349a44-2005-4aa6-9524-dbaf6f8927c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e92238b645541a9869471b308bf64c3'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:01.723611 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1426/2845] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:30:01 2022] DELETE /identity/v3/users/8e92238b645541a9869471b308bf64c3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:01.992622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:01.994139 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:02.008867 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef94b1-4627-49e9-822f-50a967ab9cb0 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.065778 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.066649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.066922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.067192 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] PATH_INFO: `/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.068385 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb2d1687b2d4b838c949bc2700b79dd)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.070293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.109808 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:02.116740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb2d1687b2d4b838c949bc2700b79dd', 'request_id': 'req-bbefbe22-621b-496d-b602-9b8c59ed0df4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:02.123363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb2d1687b2d4b838c949bc2700b79dd', 'request_id': 'req-bbefbe22-621b-496d-b602-9b8c59ed0df4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:02.130981 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb2d1687b2d4b838c949bc2700b79dd', 'request_id': 'req-bbefbe22-621b-496d-b602-9b8c59ed0df4'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:02.141169 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] The token cache is being invalidate because project 9eb2d1687b2d4b838c949bc2700b79dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:30:02.141454 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:02.142074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-bbefbe22-621b-496d-b602-9b8c59ed0df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:02.143913 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1420/2846] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:30:01 2022] DELETE /identity/v3/projects/9eb2d1687b2d4b838c949bc2700b79dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:02.148813 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.149614 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.149952 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.168211 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:30:02.169399 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:02.226812 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1427/2847] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:30:02.235433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.235723 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.235999 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.254017 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:30:02.255146 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:02.263793 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1421/2848] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:30:02.268367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.279539 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028ba5c-f2c6-4602-9117-95e10066d891 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.284223 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.284887 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.285169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.285445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.286162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:30:02.286468 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.288093 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.301011 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1428/2849] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:30:02.305253 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.314865 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297da17a-3c18-47cc-956b-4c051e4181af None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.319763 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.320412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.320700 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.321506 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.322266 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.324342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.350153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1422/2850] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:30:02 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:30:02.354655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.364449 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fe64c-18f9-45a3-9feb-6fdbf3620b38 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.377819 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:02.424937 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.425608 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.425894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.426172 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.426850 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.428803 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.448728 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1429/2851] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:30:02 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:30:02.452792 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.463847 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078831-e743-41e0-880f-15057664ce39 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.468461 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.469111 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.469391 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.469676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.470383 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.472083 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.482625 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1423/2852] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:30:02.486158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.496037 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d1504-d785-480a-97ba-fdc2c60b68ab admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.500601 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.501249 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.501533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.501809 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.502526 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.504199 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804656f-5d92-4146-bbb0-7a09419c6fa0 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.514400 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1430/2853] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:30:02.518520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.529166 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf851-e2cf-4662-b3c7-8d085f5139ca admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.533750 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.534646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.534934 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.535208 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] PATH_INFO: `/v3/projects/5868d0005f1d4fdfb929e493ad2d345d/users/75e65133aa7142eca06c814fcd9acba5/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.550690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5868d0005f1d4fdfb929e493ad2d345d, user_id=75e65133aa7142eca06c814fcd9acba5, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.553382 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.566315 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1424/2854] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:30:02 2022] PUT /identity/v3/projects/5868d0005f1d4fdfb929e493ad2d345d/users/75e65133aa7142eca06c814fcd9acba5/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:02.578004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.578481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.578602 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.606877 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] MFA Rules not processed for user `75e65133aa7142eca06c814fcd9acba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:30:02.608772 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:02.673566 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1431/2855] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:30:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:30:02.684916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:02.686620 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:02.741963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc0f2e-3080-4d46-b580-1f72d1a01c26 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:02.746159 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5524a8ebd91f4e5eb7311923401c7798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:02.746789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:02.747071 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:02.747373 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:02.755556 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:02.756931 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d8a15-88f2-4c8d-b5d1-98d947f4df6f service cinder] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:02.766096 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1425/2856] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:30:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:30:12.670689 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:12.762204 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:18.899148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:18.964438 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7a045-81de-4b37-9dbf-b95aab1170e3 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:18.968944 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:18.969808 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:18.970087 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:18.970365 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] PATH_INFO: `/v3/users/75e65133aa7142eca06c814fcd9acba5` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:18.971564 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e65133aa7142eca06c814fcd9acba5)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:18.973504 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:19.020357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e65133aa7142eca06c814fcd9acba5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.027994 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e65133aa7142eca06c814fcd9acba5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.028700 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e65133aa7142eca06c814fcd9acba5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.036723 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e65133aa7142eca06c814fcd9acba5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.037358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e65133aa7142eca06c814fcd9acba5'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.055004 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1432/2857] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:30:18 2022] DELETE /identity/v3/users/75e65133aa7142eca06c814fcd9acba5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:19.059519 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:19.060000 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:19.060099 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:19.077158 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:30:19.078363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:19.136456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1426/2858] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:30:19.146425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:19.147847 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:19.205710 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544753fc-be78-434a-a27e-c2d739e0666f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:19.209340 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:19.209983 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:19.210267 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:19.210540 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:19.219662 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:19.221481 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:19.231970 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1433/2859] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:30:19.497826 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:30:19.499299 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:30:19.513677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42305491-f64b-4638-bd3c-d4ddae3dc7b3 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:30:19.518502 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:30:19.519427 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:30:19.519716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:30:19.519988 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] PATH_INFO: `/v3/projects/5868d0005f1d4fdfb929e493ad2d345d` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:30:19.521174 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5868d0005f1d4fdfb929e493ad2d345d)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:30:19.523132 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:30:19.571645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5868d0005f1d4fdfb929e493ad2d345d', 'request_id': 'req-c903f3ec-3c87-4665-9915-67eabd59b3e9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.578206 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5868d0005f1d4fdfb929e493ad2d345d', 'request_id': 'req-c903f3ec-3c87-4665-9915-67eabd59b3e9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.585360 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5868d0005f1d4fdfb929e493ad2d345d', 'request_id': 'req-c903f3ec-3c87-4665-9915-67eabd59b3e9'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.595199 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] The token cache is being invalidate because project 5868d0005f1d4fdfb929e493ad2d345d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:30:19.595521 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.596133 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:30:19.597668 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1427/2860] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:30:19 2022] DELETE /identity/v3/projects/5868d0005f1d4fdfb929e493ad2d345d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:30:29.226896 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:30:29.592329 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:14.122596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:14.124683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:14.286202 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205efefa-cbc4-4496-9453-6349edb18531 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:14.375367 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:14.376158 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:14.376445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:14.376720 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:14.378527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:14.391553 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:14.393167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:14.405912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1434/2861] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:31:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:31:20.959936 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:20.961368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:21.089321 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903f3ec-3c87-4665-9915-67eabd59b3e9 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:21.094944 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:21.095589 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:21.095871 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:21.096145 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:21.098349 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:21.111780 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:21.113426 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:21.125655 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1428/2862] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:31:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:31:23.894930 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:23.896869 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:23.932443 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=100 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:23.995337 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149dab0-f905-4d6f-9377-2cdf886e4c56 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:23.999838 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:24.000696 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:24.000975 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:24.001246 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] PATH_INFO: `/v3/users/12845eb9ee8642b295ad4dcbe125bfd2` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:24.002408 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12845eb9ee8642b295ad4dcbe125bfd2)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:24.004345 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:24.051453 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12845eb9ee8642b295ad4dcbe125bfd2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.058252 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12845eb9ee8642b295ad4dcbe125bfd2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.058909 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12845eb9ee8642b295ad4dcbe125bfd2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.066810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12845eb9ee8642b295ad4dcbe125bfd2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.067455 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12845eb9ee8642b295ad4dcbe125bfd2'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.084238 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1435/2863] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:31:23 2022] DELETE /identity/v3/users/12845eb9ee8642b295ad4dcbe125bfd2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:24.251903 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:24.262894 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904ca1e-f61f-4a45-9ded-696ce8543a64 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:24.320899 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:24.321710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:24.321963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:24.322236 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] PATH_INFO: `/v3/projects/5513602e070b4ea09fa3b12a2b259626` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:24.323436 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5513602e070b4ea09fa3b12a2b259626)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:24.325353 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:24.374889 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5513602e070b4ea09fa3b12a2b259626', 'request_id': 'req-9d8dba8d-41c6-40db-9acc-e68c3a33787b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.381113 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5513602e070b4ea09fa3b12a2b259626', 'request_id': 'req-9d8dba8d-41c6-40db-9acc-e68c3a33787b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.389044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5513602e070b4ea09fa3b12a2b259626', 'request_id': 'req-9d8dba8d-41c6-40db-9acc-e68c3a33787b'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.393992 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:24.401919 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] The token cache is being invalidate because project 5513602e070b4ea09fa3b12a2b259626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:31:24.402630 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.403666 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.405480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1429/2864] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:31:24 2022] DELETE /identity/v3/projects/5513602e070b4ea09fa3b12a2b259626 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:24.409393 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:24.410811 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:24.423981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665105a-464f-499e-936e-22c8990fa693 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:24.479730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:24.480603 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:24.480886 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:24.481162 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] PATH_INFO: `/v3/users/85bb9c94e1914dd7988d36d00ad9116a` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:24.482341 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bb9c94e1914dd7988d36d00ad9116a)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:24.484308 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:24.505377 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:24.531270 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bb9c94e1914dd7988d36d00ad9116a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.537966 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bb9c94e1914dd7988d36d00ad9116a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.538616 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bb9c94e1914dd7988d36d00ad9116a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.546522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bb9c94e1914dd7988d36d00ad9116a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.547188 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-7465c186-d720-4080-a3e8-fb0a714601b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bb9c94e1914dd7988d36d00ad9116a'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:24.564764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1436/2865] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:31:24 2022] DELETE /identity/v3/users/85bb9c94e1914dd7988d36d00ad9116a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:24.986305 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:24.987789 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:25.002717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dba8d-41c6-40db-9acc-e68c3a33787b admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.062153 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.063036 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.063365 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.063646 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] PATH_INFO: `/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.064818 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc374d8a7cc47b3bb49b71fdb07b844)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.066747 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.112916 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc374d8a7cc47b3bb49b71fdb07b844', 'request_id': 'req-97ea8315-f549-4b25-a6f2-f50d155b5c86'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:25.119366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc374d8a7cc47b3bb49b71fdb07b844', 'request_id': 'req-97ea8315-f549-4b25-a6f2-f50d155b5c86'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:25.126412 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc374d8a7cc47b3bb49b71fdb07b844', 'request_id': 'req-97ea8315-f549-4b25-a6f2-f50d155b5c86'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:25.137052 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] The token cache is being invalidate because project cdc374d8a7cc47b3bb49b71fdb07b844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80416) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:31:25.137336 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:25.137952 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-97ea8315-f549-4b25-a6f2-f50d155b5c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:31:25.139487 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1430/2866] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:31:24 2022] DELETE /identity/v3/projects/cdc374d8a7cc47b3bb49b71fdb07b844 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:25.144598 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.145068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.145214 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.202912 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:25.204153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:25.264047 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1437/2867] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 03:31:25.273543 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da252411-8e25-49ec-bac5-76131b218154 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.274081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da252411-8e25-49ec-bac5-76131b218154 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.274081 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da252411-8e25-49ec-bac5-76131b218154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.293082 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-da252411-8e25-49ec-bac5-76131b218154 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:25.294293 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da252411-8e25-49ec-bac5-76131b218154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:25.302107 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1431/2868] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:31:25.306995 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.317082 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ddade3-4216-4b86-b268-0dda61b7d80e None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.321810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.322506 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.322791 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.323068 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.323928 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 03:31:25.324236 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.325859 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.337669 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1438/2869] 10.222.0.39 () {58 vars in 1152 bytes} [Thu May 26 03:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 03:31:25.342025 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da252411-8e25-49ec-bac5-76131b218154 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.351379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da252411-8e25-49ec-bac5-76131b218154 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.356966 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.357603 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.357879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.358147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.358824 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.360772 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.387869 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1432/2870] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:31:25 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:31:25.392301 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.403563 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81300a3-22e5-4413-9f17-837e625276f8 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.453065 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:25.461095 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.461760 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.462091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.462715 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.463048 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.465730 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.485460 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1439/2871] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:31:25 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:31:25.489755 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.500684 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd32f5-b4e8-448c-bba4-83b0782bd59e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.505379 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.506032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.506311 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.506585 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.507344 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.509488 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.520102 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1433/2872] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:31:25.524101 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.534792 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ff8cb-3797-4961-a6a1-cbb50ecfd630 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.539445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.540091 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.540596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.540596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.541373 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.542954 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b38a9-7bda-4ff0-b274-bf64517ed7e7 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.552685 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1440/2873] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:31:25.557752 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.567452 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e605-dfe5-4ccd-a9e8-c0e0fca47644 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.573049 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.573900 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] REQUEST_METHOD: `PUT` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.574178 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.574449 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] PATH_INFO: `/v3/projects/68d9565a6929495295d5344bb680a8bf/users/3955358d687743dcada3aa92ef2f8c42/roles/29fc77f10237449d82dfef29036abe51` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.589717 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d9565a6929495295d5344bb680a8bf, user_id=3955358d687743dcada3aa92ef2f8c42, role_id=29fc77f10237449d82dfef29036abe51)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.592417 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.605629 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1434/2874] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:31:25 2022] PUT /identity/v3/projects/68d9565a6929495295d5344bb680a8bf/users/3955358d687743dcada3aa92ef2f8c42/roles/29fc77f10237449d82dfef29036abe51 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:25.610627 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa4f1-d803-4894-8186-2072bc6ffb28 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.610961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa4f1-d803-4894-8186-2072bc6ffb28 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.611255 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fa4f1-d803-4894-8186-2072bc6ffb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.629444 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-971fa4f1-d803-4894-8186-2072bc6ffb28 None None] MFA Rules not processed for user `839fed83d5c84c7eb9422cc2681efe8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:25.630609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-971fa4f1-d803-4894-8186-2072bc6ffb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:25.689456 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1441/2875] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:31:25.699677 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:25.701374 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:25.726657 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:25.781831 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dfd0c-8c88-4037-a42c-8c84a9c8bdad admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:25.785535 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:25.786173 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:25.786457 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:25.786732 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:25.795887 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:25.797331 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:25.807264 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1435/2876] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:31:29.744916 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:29.745417 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:29.745521 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:29.773302 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] MFA Rules not processed for user `3955358d687743dcada3aa92ef2f8c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:29.774637 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:29.857384 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1442/2877] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:31:29.861854 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:29.876237 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37c1eb-d0ac-417b-b0ad-df137e2c37cb service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:29.881690 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:29.882363 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:29.882645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:29.882924 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:29.883659 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:29.885983 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:29.913664 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1436/2878] 10.222.0.39 () {60 vars in 1152 bytes} [Thu May 26 03:31:29 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 03:31:29.918465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:29.933240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c43b9-309d-47b5-bc62-035afc7c585b None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:29.994148 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:29.994855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:29.995132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:29.995445 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] PATH_INFO: `/v3/users` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:29.996167 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:29.998092 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:30.007225 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:30.023866 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1443/2879] 10.222.0.39 () {60 vars in 1143 bytes} [Thu May 26 03:31:29 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 03:31:30.025779 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:30.036788 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b0c3a-a481-4828-9294-0d7d151827eb admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:30.042164 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:30.042841 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:30.043270 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:30.043561 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:30.044294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:30.045978 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:30.057485 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1437/2880] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:31:30.061738 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:30.072622 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a754f-4e8b-4d31-9566-5b0a108e4e30 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:30.077436 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:30.078409 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:30.078759 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:30.079102 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] PATH_INFO: `/v3/projects/0c5e8d1fe5504ad99fdde378c11502af/users/4704b701ae2c49a19be85c1b91897032/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:30.096061 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5e8d1fe5504ad99fdde378c11502af, user_id=4704b701ae2c49a19be85c1b91897032, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:30.098655 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:30.113795 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1444/2881] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:31:30 2022] PUT /identity/v3/projects/0c5e8d1fe5504ad99fdde378c11502af/users/4704b701ae2c49a19be85c1b91897032/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:30.117819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:30.129570 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadef00-354e-4240-9ff1-cf42cdadeb98 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:30.189606 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:30.190279 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:30.190645 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:30.191088 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:30.191877 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:30.193489 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e8f99-8667-45a1-a2da-41687bf74ca9 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:30.204963 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1438/2882] 10.222.0.39 () {58 vars in 1121 bytes} [Thu May 26 03:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 03:31:30.209180 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:30.220987 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87078-e756-456c-a542-6974be84a2f5 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:30.225613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:30.226522 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:30.226804 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:30.227086 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] PATH_INFO: `/v3/projects/0c5e8d1fe5504ad99fdde378c11502af/users/4704b701ae2c49a19be85c1b91897032/roles/85e1377baa9b4d50a7aa92fe0ba16e7e` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:30.230647 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5e8d1fe5504ad99fdde378c11502af, user_id=4704b701ae2c49a19be85c1b91897032, role_id=85e1377baa9b4d50a7aa92fe0ba16e7e)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:30.234351 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:30.252533 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1445/2883] 10.222.0.39 () {60 vars in 1482 bytes} [Thu May 26 03:31:30 2022] PUT /identity/v3/projects/0c5e8d1fe5504ad99fdde378c11502af/users/4704b701ae2c49a19be85c1b91897032/roles/85e1377baa9b4d50a7aa92fe0ba16e7e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:31:34.614243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676187d-a160-499d-b9f6-693163f1fe5c None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:34.614760 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676187d-a160-499d-b9f6-693163f1fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:34.614993 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676187d-a160-499d-b9f6-693163f1fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:34.643167 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-5676187d-a160-499d-b9f6-693163f1fe5c None None] MFA Rules not processed for user `4704b701ae2c49a19be85c1b91897032`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:34.644713 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-5676187d-a160-499d-b9f6-693163f1fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:34.709207 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:34.721998 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1439/2884] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:31:34.738635 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:34.740727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:34.822981 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc956f1-1f54-465c-8b4a-b6376dfacc38 admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:34.828965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:34.829629 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:34.830190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:34.830190 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:34.888638 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:34.890240 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:34.899473 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1446/2885] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:31:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:31:44.720285 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:44.895369 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:INSERT=3,keystone:SELECT=73 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:31:50.129388 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] REQUEST_METHOD: `POST` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:50.129873 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:50.129986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:50.147819 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.auth.core [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] MFA Rules not processed for user `3955358d687743dcada3aa92ef2f8c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80416) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:31:50.149009 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:50.156045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1440/2886] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:31:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:31:50.896905 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:31:50.898496 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:50.972797 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b526f-5307-4417-a564-5fbaec6038c9 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:31:50.977645 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:31:50.978281 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:31:50.978562 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:31:50.978834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:31:50.980765 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:31:50.991986 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:31:50.993604 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:31:51.004392 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1447/2887] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:31:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:32:00.142433 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=3 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:32:01.000145 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=32 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:32:06.250038 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:32:06.251479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:32:06.308248 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66bcef-5339-4396-a66f-c5ee87561de9 None None] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:32:06.312613 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:32:06.313258 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:32:06.313544 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:32:06.313822 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:32:06.322572 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:32:06.324044 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:32:06.333404 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1441/2888] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:32:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:32:08.151613 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:32:08.153245 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:32:08.190814 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43aa49-8b16-48c1-b848-d035533ac5eb service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:32:08.196358 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:32:08.197077 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:32:08.197425 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:32:08.197764 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:32:08.200478 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:32:08.212837 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:32:08.214410 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282fb45-97da-4ec0-aef7-6d28c1579db8 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:32:08.225424 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1448/2889] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:32:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:32:16.260300 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:32:16.282590 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5aa38-0c22-43df-aebe-1ef93391fc39 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:32:16.289710 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:32:16.290601 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:32:16.290996 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:32:16.291402 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:32:16.294444 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:32:16.324673 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:32:16.326852 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:32:16.338984 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1442/2890] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:32:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:32:18.221002 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=15 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:32:26.333957 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=41 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:34:07.822885 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:34:07.823441 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:34:07.823546 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:34:07.841212 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] MFA Rules not processed for user `3955358d687743dcada3aa92ef2f8c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:34:07.842320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:34:07.849140 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1449/2891] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:34:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:34:08.152466 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:34:08.161802 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a31-af1a-48e0-a900-fc22dd4984b1 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:34:08.165456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:34:08.166085 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:34:08.166368 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:34:08.167289 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:34:08.169165 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:34:08.178860 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:34:08.180326 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:34:08.188437 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1443/2892] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 03:34:17.835843 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=3 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:34:18.184781 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:34:22.745402 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:34:22.754855 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742586e7-38e1-4b65-a003-7c3490184eea None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:34:22.758318 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:34:22.758950 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:34:22.759331 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:34:22.759536 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:34:22.761590 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:34:22.776078 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:34:22.777582 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92e124-5fea-4525-a5f7-f0b947000cf0 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:34:22.793833 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1450/2893] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:34:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:34:32.787132 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:35:21.969926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:35:21.979202 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134a7b7-3dcf-4175-a7c9-c598faefd0e1 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:35:21.982775 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:35:21.983480 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:35:21.983762 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:35:21.984037 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:35:21.985799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:35:21.995845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:35:21.997280 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:35:22.005232 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1444/2894] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:35:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:35:32.001649 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:35:59.581485 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] REQUEST_METHOD: `POST` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:35:59.582016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:35:59.582016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:35:59.599181 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.auth.core [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] MFA Rules not processed for user `3955358d687743dcada3aa92ef2f8c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 03:35:59.600390 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:35:59.607683 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1451/2895] 10.222.0.39 () {58 vars in 957 bytes} [Thu May 26 03:35:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 03:35:59.923691 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:35:59.932205 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658306-4106-40d5-84e6-7a7856ad7069 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:35:59.936342 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:35:59.936962 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:35:59.937243 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:35:59.937513 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:35:59.939261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:35:59.948552 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:35:59.950704 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:35:59.958555 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1445/2896] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:35:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:36:00.401193 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:36:00.411353 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95086e1-aa9e-423f-a8a6-36af92338d67 None None] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:36:00.414935 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:36:00.415550 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:36:00.415834 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:36:00.416108 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:36:00.418576 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:36:00.431965 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:36:00.433426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:36:00.444679 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1452/2897] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:36:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:36:09.955119 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:36:10.439435 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=9 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:36:13.115828 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:36:13.129638 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41a91b-64a9-4994-a901-3c5c7c4a7c9d service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:36:13.135389 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:36:13.135812 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:36:13.136103 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:36:13.136378 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:36:13.138197 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:36:13.151032 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:36:13.153281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:36:13.164771 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1446/2898] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:36:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:36:23.159472 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=6 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:37:24.220060 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:37:24.231327 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ade119-74c9-4cfc-85e3-284901517f26 service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:37:24.237470 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:37:24.238103 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:37:24.238383 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:37:24.238656 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:37:24.247161 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:37:24.249016 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:37:24.259026 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1453/2899] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:37:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:37:34.254758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:37:49.793926 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:37:49.803716 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef09-8551-46ee-9f1b-fb27b4e1ba8c service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:37:49.807191 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:37:49.807845 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:37:49.808121 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:37:49.808392 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:37:49.817298 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:37:49.818720 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:37:49.828144 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1447/2900] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:37:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:37:50.661261 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:37:50.678322 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1016d-da85-4ffe-827f-ac4d7080ac29 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:37:50.684758 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:37:50.685423 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:37:50.690434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:37:50.690434 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:37:50.697201 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:37:50.698796 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:37:50.710874 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1454/2901] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:37:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:37:52.832497 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:37:52.845683 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945bc20-f91d-4042-a193-a9f892c62f42 service neutron] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:37:52.852185 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9e7e21bd294ab389a68c50b4c2131a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:37:52.852922 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:37:52.853195 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:37:52.853479 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:37:52.855372 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:37:52.868361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:37:52.869946 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:37:52.881636 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1448/2902] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:37:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:38:00.705927 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=6 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:38:02.803621 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:02.813357 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c072-d9d8-4384-b2f7-0034e9f1cce1 service nova] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:02.816961 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15a31aebf6b8454cb69d574dbb0c6409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:02.817592 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:02.817873 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:02.818143 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:02.876815 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=12 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:38:02.878283 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:02.879894 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:02.889319 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1455/2903] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:38:05.928745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:05.930939 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:38:05.947640 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487112f-2a50-4e3d-aa07-9b4f2b6884fd service nova] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:05.953074 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:05.954172 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:05.954456 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:05.954729 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:05.956622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:38:05.970676 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:05.971520 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:05.984319 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1449/2904] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:38:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:38:09.007141 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:09.020967 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e5090-e0e7-4dc4-9041-cbad528f4d1f service neutron] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:09.026241 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:09.026907 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] REQUEST_METHOD: `GET` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:09.027189 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:09.027499 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:09.029320 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:38:09.043334 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:09.045210 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:09.063330 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1456/2905] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:38:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:38:09.090578 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:09.110013 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ac32-51b4-457c-b592-d1dcf1fd1b89 service placement] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:09.114622 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:09.115510 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:09.115799 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:09.116077 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] PATH_INFO: `/v3/users/3955358d687743dcada3aa92ef2f8c42` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:09.117261 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3955358d687743dcada3aa92ef2f8c42)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:09.119184 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:09.166693 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3955358d687743dcada3aa92ef2f8c42'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.167361 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3955358d687743dcada3aa92ef2f8c42'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.174986 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3955358d687743dcada3aa92ef2f8c42'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.175599 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3955358d687743dcada3aa92ef2f8c42'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.187492 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3955358d687743dcada3aa92ef2f8c42'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.199995 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1450/2906] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:38:09 2022] DELETE /identity/v3/users/3955358d687743dcada3aa92ef2f8c42 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:38:09.376783 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:09.387062 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3b8bb-377a-4668-9439-88daa5a02a09 service placement] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:09.445489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:09.446363 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:09.446648 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:09.446921 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] PATH_INFO: `/v3/projects/68d9565a6929495295d5344bb680a8bf` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:09.448153 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d9565a6929495295d5344bb680a8bf)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:09.450054 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:09.503901 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d9565a6929495295d5344bb680a8bf', 'request_id': 'req-046abd56-04d4-4a8a-a69e-75208ff9fe1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.509703 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d9565a6929495295d5344bb680a8bf', 'request_id': 'req-046abd56-04d4-4a8a-a69e-75208ff9fe1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.517527 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d9565a6929495295d5344bb680a8bf', 'request_id': 'req-046abd56-04d4-4a8a-a69e-75208ff9fe1f'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.531213 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] The token cache is being invalidate because project 68d9565a6929495295d5344bb680a8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:38:09.531609 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.532294 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.534041 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1457/2907] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:38:09 2022] DELETE /identity/v3/projects/68d9565a6929495295d5344bb680a8bf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:38:09.537879 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:09.539345 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:38:09.553127 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65eeb9-e5ba-42be-a4fc-c2e674c41ee1 admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:09.610366 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:09.611140 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:09.611453 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:09.611727 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] PATH_INFO: `/v3/users/4704b701ae2c49a19be85c1b91897032` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:09.612895 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4704b701ae2c49a19be85c1b91897032)` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:09.614846 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:09.659652 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4704b701ae2c49a19be85c1b91897032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.660294 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4704b701ae2c49a19be85c1b91897032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.667147 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4704b701ae2c49a19be85c1b91897032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.667816 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4704b701ae2c49a19be85c1b91897032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.672817 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:38:09.681898 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.notifications [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4704b701ae2c49a19be85c1b91897032'} {{(pid=80416) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:09.695251 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1451/2908] 10.222.0.39 () {58 vars in 1223 bytes} [Thu May 26 03:38:09 2022] DELETE /identity/v3/users/4704b701ae2c49a19be85c1b91897032 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:38:09.866426 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Authenticating user token {{(pid=80415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:38:09.867915 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.fernet_utils [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:38:09.881465 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046abd56-04d4-4a8a-a69e-75208ff9fe1f admin admin] Validating token access rules against request {{(pid=80415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:38:09.890486 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:38:09.942038 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '839fed83d5c84c7eb9422cc2681efe8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb9a8fcaeb99475786d754ce49440ef8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:38:09.942897 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:38:09.943178 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:38:09.943482 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] PATH_INFO: `/v3/projects/0c5e8d1fe5504ad99fdde378c11502af` {{(pid=80415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:38:09.944643 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5e8d1fe5504ad99fdde378c11502af)` {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:38:09.946580 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] RBAC: Authorization granted {{(pid=80415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:38:09.993596 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5e8d1fe5504ad99fdde378c11502af', 'request_id': 'req-1aae7b21-56c8-40ef-8bb7-703e28395b38'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:10.000169 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5e8d1fe5504ad99fdde378c11502af', 'request_id': 'req-1aae7b21-56c8-40ef-8bb7-703e28395b38'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:10.006224 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5e8d1fe5504ad99fdde378c11502af', 'request_id': 'req-1aae7b21-56c8-40ef-8bb7-703e28395b38'} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:10.018810 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] The token cache is being invalidate because project 0c5e8d1fe5504ad99fdde378c11502af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80415) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 03:38:10.019428 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:10.019727 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG keystone.notifications [None req-1aae7b21-56c8-40ef-8bb7-703e28395b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80415) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 03:38:10.021489 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: [pid: 80415|app: 0|req: 1458/2909] 10.222.0.39 () {58 vars in 1232 bytes} [Thu May 26 03:38:09 2022] DELETE /identity/v3/projects/0c5e8d1fe5504ad99fdde378c11502af => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 03:38:19.691748 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:38:20.015959 dv-cinder-843361-1-iscsi devstack@keystone.service[80415]: DEBUG dbcounter [-] [80415] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 03:42:28.204091 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Authenticating user token {{(pid=80416) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 03:42:28.205494 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:42:28.343045 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d266dd2-abd4-4f6b-b62f-448c2544f25e admin admin] Validating token access rules against request {{(pid=80416) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 03:42:28.403067 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c33cd0ddf4fc49e0b8e9d6f0863f81dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c16d6b8e11b44d9a961a2be0a71b1e74', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80416) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 03:42:28.403745 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] REQUEST_METHOD: `GET` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 03:42:28.404027 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] SCRIPT_NAME: `/identity` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 03:42:28.404302 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80416) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 03:42:28.406075 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.fernet_utils [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80416) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 03:42:28.417177 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 03:42:28.418740 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a0981-8880-436b-a930-86c49d0bf862 service placement] RBAC: Authorization granted {{(pid=80416) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 03:42:28.429123 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: [pid: 80416|app: 0|req: 1452/2910] 10.222.0.39 () {60 vars in 1365 bytes} [Thu May 26 03:42:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 03:42:38.424281 dv-cinder-843361-1-iscsi devstack@keystone.service[80416]: DEBUG dbcounter [-] [80416] Writing DB stats keystone:SELECT=89 {{(pid=80416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}